You have to look at how they ask a question, most of the time you have to answer in the same tense used in the original question.
eg: You (do) speak English ?! Don't you ? "(do)" it is not necessarily needed
eg: You want to go ? Don't you ?
eg: You don't speak English ? Do you ?
eg: You don't want to go ? Do you ?
So, looking at these sentences you can see that when you start with a negative question the tag (Do you ?/Don't you?) at the end has to be positive. When you start with positive question the tag has to be negative.
Positive question -> Negative Tag (Don't you?)
Negative question -> Positive Tag (Do you?)
Personaly, I think, that starting with a positive question and ending with a negative tag is the easiest. The other way around sounds a little bit strange, "clumpsy" to me too. I mean I do have to pay attention to it myself but the moment I use a negative question it's like an alarm going off in my head just don't want to make a "tag mistake"