Yes. We have been using it so. Trouble is, there ARE linguists and teachers who are teaching that we should say in answer:
B. Yes, I ate lunch in the afternoon.
B. Yes, I ate lunch at three.
B. Yes, I visited Dalian in July.

1. This is the matter that has got some people upset (never mind mentioning whoever), because it is or can be confusing.
2. The "rule" is not my view. I did some checking and found that that's what is being taught by SOME. I don't know if it's a majority or minority who subscribe to the "rule", or it's at the climatic state of a toss-up!!!!!
3. It seems the language has gone through "refinements" (for whatever reasons???). Without sounding disparaging, it appears that such refinements are the labour of love of purists.
4. I guess you and I grew up speaking quite a bit of unsound grammar in pure blissful ignorance!!!!
5. I'm more concerned with the practical side of things: With such differing views, what should we teach learners? And it could lead to a lot of angst in the poor serious-type of kid (not to mention even some adults).