In my opinion, yes. However, you need to be patient. You won't understand everything immediately but that's not important. What is important is to "open your ears" to the sounds of the language - the rhythm, the stress etc. The longer you listen at any one time, the more used to it you will get.
If you have English-language DVDs and there's an English subtitles option (sometimes available for the hard of hearing), a great way to learn is to have the subtitles on, sellotape a flap over the subtitles, listen and then lift the flap to see how closely your understanding matches the subtitles. It's a great way to focus on the spoken word and you'll soon be seeing your success rate shooting up.
(I myself watch English-language films with subtitles on but that's because my TV is ancient and the sound is awful!)