I am not a teacher.
Both are possible. The choice between 'going' and 'coming' usually depends on where the people actually are.
I don't understand where your neighbour is when this conversation is taking place. Are you speaking to him on the phone?
If he's with you here (wherever that is) then I would use 'going'. If he's there but you're not, I would use 'coming'.