As Piscean said, sometimes "yesterday evening" and "last night" are interchangeable but not always. For me, it depends on the duration of the event. If I did something from 7pm until 8pm, I'd say "yesterday evening". If I did something from 7pm until (or until after) midnight, I'd say "last night". If I did something from 10pm until 11pm, I'd say "last night".
If I did something from 7pm until 9pm, I'm not sure which I'd choose. It might depend on how light it was! In the height of summer, when it's light until nearly 10pm, I'd probably use "yesterday evening" but in the depths of winter when it's dark by 4pm, I'd probably use "last night". Many of us associate night with darkness.