'By the time I come back, I expect you to have written at least a paragraph.'
There is another use, disapproved of by some, where the infinitive refers to an action already completed.
'Darwin was the first to formulate a theory of evolution, but he is not the first to have used the phrase 'survival of the fittest.'
This would be better: '... he was not the first to use...' . What would certainly not be better is the form '... he was not the first to have used...'.
Another use that refers to a completed action is this (also not accepted by all):
'Is Samantha the only one of you to have seen Casablanca?'
I'm not sure whether to call this 'wrong', but I'd use 'the only one who has seen', to avoid arguments with grammarians who have strong views.
Student or Learner