Interested in Language
I went to your office yesterday, but you weren't in.
Why not "came"? The movement was towards the hearer or more precisely where he was expected to be.
I will come to you if you like. Speaker ==> Hearer
Will you come to see me too? Hearer ==> Speaker
You'll come to me. Hearer ==> Speaker
Does it work differently in the past? It seems that in the past it's not only about the speaker-hearer movement but also about the speaker's positioning.
If the speaker an/.or listener are in the listener's office at the moment of speaking, then the speaker is likely to say 'came'.
If neither is in the listener's office, thn the speaker is likely to say 'went'.
However, there is no 'rule' about this - it's a matter of personal perception. So, if I mentally associate the listener with always being in their office, I might well use 'came' even if neither of us is there at the moment of speaking. This explains why we sometimes hear people use the word we would not normally expect.