I prefer it with "that" solely because it tells the listener that the sentence isn't just going to be "He suddenly realised what he had done". That is a perfectly grammatical sentence but, by adding "that", we know that something more is to come after "what he had done".