Yesterday evening he came to dinner.
Why is it wrong to say,
Yesterday in the evening he came to dinner.
It isn't, especially if you put commas round 'in the evening'. However, it's not nearly as natural as the first sentence.
He came to dinner yesterday (evening) is probably the most natural.