Ah! Now I got it
My Oxford Advanced Learner's only says:
on the contrary: used to introduce a statement that says the opposite of the last one.
And that was the only definition I knew, before I checked The American Heritage, which says:
on the contrary :In opposition to what has been stated or what is expected.
I've learned a lot now. :D
Thank you!