'yesterday morning', 'yesterday afternoon', 'yesterday evening' are all possible except 'yesterday night', which is an exception. Koreans also make a mistake of saying 'yesterday night' as the Korean language has the same composition or word order. So it should be always "last night", right?