The only time I could not understand a native speaker of English (who was sane and sober) was in Scotland. Cajuns are also tough.
Many time I may miss a word or two from other people with an accent very different from mine but only that one time was I left mystified over the entire conversation. Fortunately it was one taking place next to me not with me.
Student or Learner