One factor that makes English seem an easy language is the fact that it is ubiquitous. It's simply impossible not to have some sort of contact with the language wherever you're located in the world. Ninety-nine percent of the movies that are now showing in the cinema are in English. We also listen to a lot of songs in English, and have loads of English TV-series being broadcasted.
The impression we have is that it is easier than, say, German, because that one is not a language we see on a daily basis.
I guess I could say I am able to speak English fairly well, and I am now learning German. I'm a beginner at German, and it's definitely not easy. Loads of people say it's more difficult than English, but, is that so?
To start with, I think that's a very ambiguous question. In what sense is German harder? Learning German has shown me that the English spelling can be quite a hurdle (it's a lot harder than Portuguese, my mother-tongue, a language which many native speakers can't write properly). On top of that, I can understand German speakers on Youtube much better than I could English when I was at that level in it.
Thus, my answer is NO: English is definitely not an easy language TO MASTER. As most people reach a basic level in it (which is radically easy compared to an advanced one), they claim it's an easy language even while making loads of mistakes when speaking. But very few people do aquire an advanced level. That shows that it is indeed a hard language.