They're so alike that I'm often misunderstand them
The same happens to were and weren't. For example: somebody says" It wasn't me" I often interpret as "it was me"
Speakers often do not release the [t] in wasn't and weren't but they do pronounce the [n], so stop focusing on listening for [t] and start listening for [n].