1-Your attention to what he says and not to what I say is annoying.
2-Your kindness to him and not to me is annoying.
3-Your politeness toward him and not toward me is annoying.
They are grammatically correct
4-Your anger at me and not at him, since we were both equally responsible for the accident, is annoying.