Teachers not knowing the answers (without checking the key)

[seanog]

Hi, I was wondering what the opinions of other teachers are on this point: In an advanced exam like IELTS, I think, in all probability, the only person who could get all the answers correct first time, especially in a listening exercise, would be whoever set the questions.
Normally of course it's very easy for teachers who systematically prepare classes and look at the answer key before classes.

In fact, I think those questions, which an educated and competent native speaker does not answer correctly, should not even be in the exam.
It would be more appropriate to include more phrasal verbs or idiomatic expressions or similar instead. So, if experienced teachers can't answer all these questions correctly, what chances have average native English speakers got? Any opinions?

[RonBee]

It is certainly true that there can be a difference of opinion as to what is and isn't correct.

[seanog]

For example I imagine whoever sets the listening tests at advanced level spends quite some time poring over the text, trying to think of ways to make it difficult for the student to get the right answer. Whereas the student, in some cases only has one chance to listen, and must answer within a short period of time.

It is even possible that a learner who spends many hours doing listening exercises, could get more answers correct than her or his teacher would have got, but this obviously doesn't mean that the student's overeall level in listening is higher than the teacher's - the teacher would certainly understand all (or nearly all) of the many idiomatic expressions, phrasal verbs, colloquial expressions etc...in "real world" listening, unlike the student.

Similarly, some of the reading comprehension questions would be very hard for a competent native speaker or even an experienced teacher, to answer correctly.
I really think something isn't right with the testing system when probably most competent native speakers would either misunderstand the questions, or for whatever reason, be unable to answer all the questions correctly.
After all, any student would be thrilled to pass for a native speaker, so tests should have that objective: to give the highest marks to those students who most closely resemble native speakers.

[Raymott]

Originally Posted by seanog

I really think something isn't right with the testing system when probably most competent native speakers would either misunderstand the questions, or for whatever reason, be unable to answer all the questions correctly.
You are correct. There is often something not right with the testing system. We get ample evidence of this from test questions posted here. (Not specifically IELTS). You must remember that the tests are set by imperfect and not necessarily overly-intelligent or insightful humans.

After all, any student would be thrilled to pass for a native speaker, so tests should have that objective: to give the highest marks to those students who most closely resemble native speakers.
But if that were the criterion, the test-setter would need to know
what answer native speakers preferred. It is easier for a teacher to set a test on the basis of their own knowledge of grammar and usage. Ideally, all tests should be checked by two other competent speakers and if there is any disagreement on a question, that question should be thrown out. But who's going to pay for all this work?

R.

[Tdol]

One things to note is that many exams have a number of set possibilities for correct answers to some questions, so while I agree that teachers should know the correct answers, answer keys may help supply all possibilities in these cases. For example, there may be a regional form that is in wide usage but regarded as non-standard, so an answer key would let the teacher know whether this form would be acceptable. There are differences between exams about the acceptability of forms like 'If I was', for instance. However, if a teacher needs answers for everything, they are probably in the wrong job.

[seanog]

It would be an interesting experiment if 100 teachers (ok, let's say experienced and "highly" qualified teachers) were to sit the same exams as their students, how many of them would get all the listening and reading comprehension questions correct? I am referring to advanced level, and higher. At a guess I'd say that in the listening not one would get ALL the answers right, and in the reading comprehension, for example combining titles with paragraphs etc, I would be surprised anyone got all of them right either.

This is not to in any way cast aspersions on teachers - far from it. I'm just making the point that the testing system seems to be inadequate. For example, maybe a specific student would get better marks than her teacher in listening, but if both of them went to see any recent movie, I'm quite sure the teacher would get a far higher mark for that "listening" test.

To be honest I'd say a much more accurate listening test would be a straight-forward dictation where the students write down what they listen to, eventhough I realize that multiple choice questions are a lot easier (and cheaper) to correct.

[Alex Case]

I've taken CAE, CPE and IELTS listening tests to judge which bits my students would find difficult, and though I've had to concentrate I've got every question right. On reading, I usually get a question or two wrong but can instantly see that it was just a silly mistake once I look at the answer key. I have never seen a question in official exam papers for these exams that are debateable, though they might exist. TOEFL and TOEIC are different as a second of lack of attention can throw you off completely as the questions don't come till afterwards and so you don't know what you're looking for. Even so, I get very few answers wrong as long as I force myself to concentrate.

Writing is another matter- many not particularly well educated native speakers could fail IELTS writing. I've also given near native speakers low scores in BULATS speaking for not answering the question, only very short answers and minimal vocabulary use.

[seanog]

All very well. But how easy is to to fully concentrate in the tense atmosphere of of an advanced exam? Or maybe there could be an exceptional case where a particular listening test is less confusing than normal, and a very highly focused native speaker would get all the answers right. But it's worth bearing in mind that teachers are normal people and can make mistakes like anyone else.

[Alex Case]

Basically, a native speaker getting answers wrong or needing to really concentrate is the sign of a bad test that is testing things other than listening comprehension, e.g. short term memory.

[seanog]

I agree. This is exactly the issue. I think a more reliable way of testing listening would be by doing a dictation test; 20 or 30 sentences could be read out, using multiple choice options for each sentence.
And I'd say some of the reading comprehension questions would be utterly confusing to many native speakers too, especially having paragraph titles all jumbled up.

But yes, a good criterion would be to consider how difficult, or confusing, an advanced test might be for competent native speakers.

And I doubt very much whether a non-native speaker who gets top marks in the most advanced exam, would ever reach the level of a native speaker with "bad" grammar.