Though I know that adverbs of frequency go after be, my fingers wouldn't type 'was never' I've seen 'never was' so often that the phrase must have crept into the unconscious. Thanks for pointing that out!
Try to relax - when you are speaking informally at least. Many of the things that we note as 'incorrect' in this forum would pass unremarked when you are speaking.
Take your two sentences, for example. Even if you uttered one in a situation in which 99% of native speakers would utter the other, the meaning would still come clearly through. And if you said 'never was' when 99% of us would say 'was never', very few people would even notice.
You want to speak and write as naturally and accurately as possible. That is natural, but try to enjoy your efforts.