There is a very simple answer to your question - practise! Many people, even with a high level of a foreign language, have trouble understanding all the dialogue in TV shows and films. This is usually due to the fact that the actors speak quickly and colloquially, they might use slang terms.
My parents both speak fluent French. Recently, they both watched a French series and decided not to read the subtitles. My father understood, he says, 99% of it (partly because the series is set in the region of France where he learnt French from the locals). My mother found herself reading the subtitles quite frequently. She said that was due to the speed of the speech, the use of colloquialisms and the mild regional accent.
You need to listen to and watch as much English as you can. Find an English-speaking radio station and have it on all the time, even if it's just in the background. A station with mostly talking and not much music is best. You don't need to sit and concentrate on it. Just be aware of the rhythm and cadence of the language. Try watching a TV show or film with the subtitles on and then a few days later, watch it again with no subtitles. Of course, you'll be able to remember some of the dialogue from the subtitles but you will probably find that you hear/understand more of it.
There is no quick fix to this, though.
Student or Learner