[CAUTION: I am not a teacher:take the advice and or corrections offered in this post at your own risk.
If you doubt the information, please get a qualified opinion from one of the teachers on these forums.]
One crucial word has been omitted:"...at this same time tomorrow", as it often is because this expression is already understood to mean "at this (same) time".
The speaker is probably referring to a time, as referenced by a clock (say, two o'clock) or could also be referring to a more general time(as in "break-time").
Time, in the sense of time of day, as in hours and minutes, is never countable. Do not be confused by something like: "at three separate times during the day". This refers to events, which happen to be measured by time of day (or not:dawn,noon and dusk for example)
"Times" refers to the number of incidents of an occurence or action. (He tapped three times) or the more general sense with which we refer to "changing times". As in the magazine "Times" and the public square "Times", neither of which, in my private opinion, make much sense (imagine a magazine called "Peoples"). Their notorious omission of "the", or any definitive article may have contributed to your confusion.