Use of the verb to watch

[petronio]

I am from Brazil

I was having a conversation with a group of english people and they asked me: "What are you going to do tomorrow afternoon." I Answered: "I am going to watch an english class." They told me that I couldn´t say "to watch class" because it is wrong but they didn´t give me an explanation.
According to The Oxford Advanced Learner´s Dictionary the definition of the verb to watch is: to look at somebody or something for a time, paying attention to what happens. According to the definition above I can say to watch class, is the dictionary wrong?

[Red5]

Welcome!

I think they might have been expecting you to use the word observe instead of watch.

[RonBee]

The person watching the class would most likely be somebody making sure they didn't get into trouble.

[Dandelion]

You could also say you are going to sit in on an English class. That means that you will be present but will not actively participate in it.