It's an outdated usage of "mind". These days, we say "you didn't listen to me", "you didn't take any notice of me" or, as you said, "you didn't pay attention".
I would suggest using the past continuous in both halves of the sentence.
I was talking to you but you weren't listening [to me].
I was talking to you but you weren't taking any notice [of me].
I was talking to you but you weren't paying attention [to me].