Was the exam hard? <--- correct
Was the exam difficult? <--- correct
I had to work really hard. <--- correct
I had to work really hardly. <--- incorrect
I had to do some difficult work. <-- correct
hardly in this context would means "almost nothing" or "very little" and would not work with "really hardly" or in this sentence in general.
If you wanted to say you did very little work you could say:
I hardly had to work.