Common sense tells me when using the present tense, I should say:

He lets me use his pen.

However, I always want to say:

He let me use his pen.

Which is correct, and which is the one people use in everyday conversation? Thanks!