I have learned the use of Direct object and indirect object. But I've got confused with this use. Please check my knowledge.

He gives me a book.
He gives a book to me.

The second sentence is said to be wrong( according to a book).
It is said "to" should not be used before "pronouns", but should be used before "nouns".

Please explain.