Student or Learner
It is not necessary, however, for the past tense to be accompanied by an overt indicator of time. All that is required is that the speaker should be able to
count on the hearer's assumption that he has a specific time in mind.
(A Comprehensive Grammar of the English Language)
Have I understood the part in bold right?
'the speaker should know that the hearer thinks that the speaker has a specific time in mind'
Yes, you have understood the part in bold correctly.