I'd like to ask about the meaning of "drag you down with them".
If you hear a rumor about something that has been said about you, make a joke out of it. Shrug it off and don't let it upset you into doing something foolish. Don't start your own retaliatory rumor; sinking down to your opponents' level will only drag you down with them.
I looked this phrase up in the Oxford advanced learner's dictionary:
drag sbdown to make sb feel weak or unhappy: Dwelling on the past only drags you down.
But I think that in this case the phrase "drag you down with them" doesn't imply that one will feel weak and unhappy. It seems to me that the phrase means that if one starts one's own retaliatory rumor one will become as bad/immoral person as his/her opponents. On the other hand, I'm not sure in my own interpretation.
Your interpretation is pretty good. In fact, in this case it is spot on.