No one else has tackled this question, so I will, though I'm afraid my answer might not be helpful.
I wonder if there's some misunderstanding here. The two sounds that confuse you are very different. I don't see how you could have trouble hearing the difference.
a: is similar to the vowel in German "Vater."
ɔ: is close to the vowel in German "Sorge."
You won't get much help from the spelling as to which pronunciation occurs in any given word.
I wonder if you're confusing British vowels with American vowels.
In British English "father", "not", and "all" have three different vowels.
In North America, these words are usually pronounced with the same vowel.
Good luck to you.