The first sentence makes sense as it stands, and means that he is still employed in that new job.
The second sentence does not make sense as it stands. It can only form the first part of a sentence saying what happened next. For example: "Finally, he got a new job, which he had been doing for the past two years when he was abruptly sacked last week."
Student or Learner