I'd say that you can't really tell. The only thing you know is that John had that habit in the past, and that he no longer does. but the sentence contains no specific information about three years ago. He might have already stopped three years ago, or he might have quit a month ago)
Three years ago, I didn't know that John used to smoke.
Does this mean:
1-John used to smoke three years ago
2-John had already stopped smoking three years ago
3-We can't tell
Three years ago I wasn't aware of the following information: John used to smoke (i.e. Although I didn't know it at the time, when I met John three years ago, he had given up smoking--he used to smoke--before I met him.)
1- John used to smoke three years ago. (incorrect)
==> Three years ago modifies I didn't know X.
2- John had already stopped smoking before I met him three years ago. (correct)
==> According to the statement, I was not aware that John used to be before I met him.
3- We can't tell. (incorrect)
==> We know. It's the position of the adverb phrase "Three years ago" and the fact that it modifies the main verb "I didn't know". That John 'used to" has nothing to do with 'three years ago.'
I didn't know , three years ago, that John used to smoke.
In other words, at the present, I am aware that John doesn't smoke. I've known him for three years. Three years ago, when we first met, I was not aware of the fact that he had given up smoking, that he no longer smoked, that he used to be a smoker. He never mentioned it to me. I just now heard someone say,
Someone: John used to smoke, you know?
Me: Three years ago (when we first met) I didn't know at that time that John used to be a smoker (i.e. had given up smoking).