You can say It won't take me longer than an hour, for example. "It won't take me" does not work with "later". Tdol's suggestion in post #6 is good. In conversational American English you might say I'll be home by five at the latest.Is it incorrect to say "By 5 PM at the latest"? No, that's OK.
What is the proper way of saying this? "It won't take me later than 5 PM to reach home"?