They mean completely different things.
To "answer to" (or "be answerable to") someone means that you are responsible to that individual for something. Thus, you answer to your teacher for the quality of your work. Or your boss could say to you "You answer to me for being on time every day".
To "answer" someone is to reply to their question. The only time "Answer to me" might be used in that sense of "answer" might be if a class of pupils were asked a question by one teacher, but another teacher wanted the responses to be made to him or her.
Student or Learner