Hello ibrahim and welcome to the forums.
I cannot agree with your interpretation.
The original sentence does not say a word about who the person associates with. It simply says that he (or she) sees people smoking at some point almost every day. The person may despise smoke, hate the way people who smoke smell, and refuse to pass his (or her) time with them, and see people smoking through the window or in the distance.
As a "By the way" comment, I don't find the original very natural.
I'd prefer any of these:
Hardly a day goes by that I don't see someone smoking.
Hardly a day goes by when you don't seem someone smoking. (The impersonal "you.")
Hardly a day goes by that one will not see someone someone smoking.