There are at least two meanings for this idiomatic phrase.
1. Good walls that keep out sound stop me from hearing my neighbour and so my neighbour will not annoy me and the reverse will be the case as I will not annoy my neighbour so we will not fight. We will be good neighbours.
2. Good walls keep my neighbour away from me despite the intentions of my neighbour so even if my neighbour is a dead slime I don't care because my good walls keep my neighbour away from me and I am happy with the situation.