What if 94% of teachers are better than average?

In its ceaseless strip-mining of institutional humour, Dilbert the other day dug up the old line about 74% of managers thinking they were above average. This is only one of a large collection of such factoids, usually (and probably justly) trotted out to illustrate that overconfidence goes hand-in-hand with expertise. The equivalent in academic circles is the claim that 94% of “college professors” believe that they are better than average at teaching, which appears to go back to a paper by Cross (1977); sadly I don’t have access to the full text so I’m having to go by hearsay. I’m not going to argue against the general conclusion that expertise breeds overconfidence, and I don’t find it implausible that American professors — a species not noticeably short of self-confidence — might be peculiarly prone to this condition. (There’s an interesting Wikipedia page on “illusory superiority”, though it doesn’t seem to cover what I’m going to say here.) I would like to argue that something can be said in defence of these professors.

I’m also not going to argue about the meaning of “average”: let’s assume that the relevant average in this case is the median, so that for performance measured on a given scale, 50% of the population should be above and 50% below average. My argument is that unless one’s assumptions are very carefully defined, a question like “are you an above-average teacher?” will be interpreted by each respondent according to their own standards — and these standards may differ considerably without being either self-serving or unreasonable.

Suppose that the population we’re interested in comprises three lecturers. Lecturer A believes that the quality of her teaching can be gauged by the proportion of her students who pass the exam and are thus able to proceed with their study. Lecturer B believes that the quality of his teaching can be gauged by how enthusiastically the students engage with it. Lecturer C believes that the quality of her teaching can be gauged by how many of the students are provoked to explore the subject beyond the confines of the syllabus. Consequently, Lecturer A conscientiously drills her students in exam-type questions: they don’t especially enjoy this, but every year they achieve a 95% pass rate, and many of them continue with the subject because they now feel they know how to do it. Lecturer B puts immense effort into his classes: he’s a lively speaker who bubbles with enthusiasm, deploys multimedia and pop-cultural references, and can even tell a decent joke. His teaching evaluation questionnaires are a joy to read; his pass rates aren’t bad, and some students do get more interested in the subject than in the presentation, and can survive a more arid treatment of it in later years. Lecturer C is always looking out for ways to challenge her students. She never sets two questions that draw on exactly the same knowledge; she refers them to alternative treatments of each topic; she digresses both into the foundations and the applications of the subject. Her teaching evaluation questionnaires complain that she wastes time on material that won’t be in the exam, and that she’s only interested in the best students; when it comes to the exam some students seem completely bemused, and the pass rate is sometimes abominable. Ten years later, though, a handful of her students are still in touch with her and with the subject; a few have gone on to PhDs; one or two have already managed to teach her something new about it.

Lecturers A, B and C are, of course, caricatures, but not very gross ones. The point is that, by their lights, each is doing a better job of teaching than the other two. Lecturer A feels secretly that B is a bit of a charlatan and C a bit of a snob, and that both are letting their weaker students down. Lecturer B feels secretly that A and C are the reason that lecturing has such a bad reputation — surely it can’t be so hard to communicate in an engaging manner? And Lecturer C feels secretly that A and B are just going through the motions, and that no real mathematics ever takes place in their classes. Each of these secret beliefs, frankly, has something to be said for it — yet a survey would show that 100% of these lecturers think they they’re above average, and so at least one or two of them must be deluded. To complicate the picture, if A, B and C work in a large and wise department, it will probably have deployed A to service classes, B to first-year introductory classes and C to specialised final-year classes, so none of these lecturers will see a good reason to change their views.

One could try to square the circle by saying that a really good lecturer would combine the best features of A, B and C, but this seems doomed to the usual fate of quadrators. If we’re looking at priorities, something has to come first: class time spent on digressions will not satisfy the student who wants exam practice or vice versa; and popularity and challenge are rarely truly compatible. (Gilbert Highet’s classic The Art of Teaching, famously, cites Jesus and Socrates to make that latter point.) At some point, each of us has to ask, whether at the level of an individual class or at that of our whole teaching “philosophy”, not only who has the right to judge our teaching, but by what standards it ought to be judged.

So where does that leave me? By A’s standards, I’m certainly not above average, as the pass rates in my classes indicate; I’m elitist enough not to be worried by this most of the time, though my conscience twangs now and again when I think of some of my more amiable-but-witless students. My inclination probably calls me more towards C’s standards of assessment, but the evidence from questionnaires suggests that my best claim to beating par might be to judge myself by B’s standards. And can I claim to be above average by the standards I’d prefer? The simple answer is that I don’t know. Maybe after only a few years of lecturing and a couple of thousand students, it’s still too early to tell. Maybe it will always be too early to tell, and the most honest response to that 1977 survey question would have been either “it’s a stupid question” or “I have no idea”. Are these options, I wonder, ever included in such surveys?

This entry was posted in Teaching. Bookmark the permalink.

3 Responses to What if 94% of teachers are better than average?

  1. Brilliant discussion of a stat that is often tossed around as true. I am reminded of the book, Lie With Statistics. I found you in my search for that very stat in use of an article I’m working on about excellence. “94% of teachers are better than average?” I appreciate the humility you bring to the discussion. I agree, it all depends on what you are measuring. And I love the options on the survey of, “It’s a stupid question.” or “I have no idea.” Good work, guy.

  2. Pingback: Is this the best essay ever written about university education? | New-cleckit dominie

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s