Only capitalize these words if they are being used as the name of the person (just as you did in your example). It would not have be in direct address, however. For example: We always loved our visits to Grandmother, but: We always loved visiting our grandmother.
I hope this is helpful,
- For Teachers