[Mike the cool person]

A simple way of explaining this is to talk about "one year's jail". If it's one year in jail, why is there an "s" on the end of year? It's not there to signify that it's plural, so it must be there to signify possession. In which case it does have an apostrophe. So, "five years' jail" is correct.