NOT A TEACHER
In your sentence: the referee told the player to leave the field, probably because he violated the rules of the game.MCGraw-Hill Osborne Dictionary of American Idioms and Phrasal Verbs:
order someone off ((of ) something)
to command someone to get off something. (Of is usually retained before pronouns.)
The teacher ordered Tom off the steps.
He ordered him off.