I am going to be with John.
could never carry the idea of physical motion.
But can't one say:
A-I went to be with John.
I think that might be correct. It could have no meaning other than that.
I think people use
'went to be with the lord'
often as a euphemism for 'died'.