Marked by a stranger

For most of the year, the person who marks your child’s English knows how your child thinks. When a clumsy sentence turns up in an essay, the teacher knows what your child meant by it. When a paragraph about the book comes out thin, the teacher knows your child read the whole book and reads the paragraph generously. All of that knowledge quietly shapes the mark.

Then, one morning in May, your child sits the exam, and the writing goes to a stranger.

The examiner has never met your child and never will. Your child’s script is one of possibly hundreds that this one person will work through in that series, inside a marking window in which one exam board, AQA, handles more than seven million scripts.^[6] None of what the teacher knew travels with the script, so the stranger has only two things to go on: what appears on the page, and a document setting out what earns marks.

Those two readers, the teacher and the stranger, want overlapping but different things. Most students do not find out how different until it is too late to matter. A student who has pleased one reader for years can still struggle with the next.

Ask tonight what the examiner rewards

You can try this at the dinner table in about a minute.

Ask your child how their English exam is marked. You are not asking what is on the paper, but how the marks are given out. What does the examiner look for in a paragraph of analysis? What separates an answer near the top from one in the middle, and how many marks go to the writing as opposed to the reading?

We ask a version of this of every student who joins us, and the answers are remarkably consistent across strong students, weak students and students at excellent schools. They can tell you the topics, and they can tell you what their teacher tends to say. Very few can describe, with any precision, what the examiner is actually rewarding.

Parents are often surprised by that answer. The surprise fades once they notice who has been reading their child’s work all year.

Years of pleasing one reader

When a teacher is the reader, a student learns to satisfy the teacher. That is sensible behaviour, and it usually works because teachers are consistent, explain what they want and mark generously when they can see the understanding behind an awkward sentence.

So a student builds up an inner picture of “what good work looks like” that is really a picture of what this teacher responds well to: highlighted quotations, longer answers, the technique named, a neat conclusion.

Much of that picture overlaps with what the exam rewards. Some of it does not, and while a teacher does the marking, those parts stay invisible to everybody.

Then, in May, the reader changes without warning.

You cannot improve work you cannot judge

What would a student need in order to get better at writing on their own, with nobody marking as they go? The clearest answer comes from an Australian researcher, Royce Sadler, in a 1989 paper that has shaped how assessment is understood ever since.^[1]

His argument is simple to state. For a student to improve their own work, they have to be able to judge its quality while they are producing it. That means holding a picture of good work that closely matches the picture the marker holds. Without that picture the student is writing in hope, and the only way to find out whether a paragraph worked is to hand it in and wait for the mark.

A student in an exam hall gets no chance to hand anything in and wait. They have to make that judgement themselves, in the moment, about their own paragraph, alone.

Why the mark scheme alone fails

The obvious response is to print off the mark scheme and give it to them.

Every school in the country has already done this. The criteria are published, put up on classroom walls and handed out before mocks. Almost all the students who cannot describe what the examiner wants have been shown the document: they have held it in their hands and still cannot say what it asks for.

Sadler explains why the document on its own is not enough. Standards are partly tacit. They cannot be fully captured in words.^[1] A mark scheme is a summary of a judgement rather than the judgement itself. Phrases like “perceptive”, “judicious” and “convincing” mean something precise to an examiner who has marked a thousand scripts, but to a fifteen-year-old reading them for the first time they are just approving adjectives.

Being handed the criteria is like being handed the rules of a sport you have never watched. Nothing in the wording tells you what the thing actually looks like.

Students learn a standard by using it

Researchers have tested this directly. Chris Rust, Margaret Price and Berry O’Donovan ran a study on exactly this problem, and found that publishing the criteria produced little on its own.^[2]

What produced measurably better performance was different. Students were given real pieces of work, asked to mark them against the criteria themselves, and then made to discuss and defend the marks they had given. Arguing about a judgement is what transfers the part of the standard that cannot be written down.

The wider evidence agrees. Teaching students to monitor and evaluate their own work is one of the strongest and cheapest things education research has found, worth around seven months of additional progress.^[3] It also works best when it is taught inside the subject itself rather than as a separate study-skills lesson.^[4]

So the skill can be learned, but a student learns it by making judgements and defending them, never by reading a list of criteria.

Memorised instructions break under pressure

One more finding explains the results that seem to make no sense.

Psychologists who study why people underperform in high-pressure moments have looked closely at working memory, the small amount of information a person can actively hold in mind at once. Under pressure, worry takes up part of that space, and whatever the task needed the space for, it no longer has.^[5]

Who suffers most? In these studies, it is often the people with the greatest working-memory capacity, because they rely on methods that depend on holding a lot in mind at once. When the pressure takes the capacity away, their method goes with it, while students using simpler, more automatic approaches are less disturbed.^[5]

Now apply that to an exam hall. A student whose whole method is remember what the teacher said we should do is running on the exact resource the pressure uses up. A student who understands what is being rewarded has something sturdier to fall back on: an understanding of the task, rather than a memory of instructions.

We think this is where many of the disappointing results come from. They come less often from lazy students than from strong ones, in good schools, who did everything asked of them and had nothing underneath it when the room went quiet.

What the research shows, and what it does not

First, this research was not done in exam halls. Much of it involves maths problems in laboratory conditions, and the step from there to GCSE English is an inference. We think the step holds, but it is our own inference rather than a finding.

Second, every August you will see parents on social media whose children did well all year and came home with a grade five or six. The pattern is real and widely observed, but we could not find a sound British statistic measuring it, so we are not going to invent one. Treat it as something to recognise rather than something we can count.

Three things to do this term

1. Ask the question from the top of this page. If your child can only answer in general terms, such as “you have to analyse the language” or “you need quotations”, you have found the gap now rather than in August.

2. Have them mark a real answer against the real standard. Reading the criteria again will not help much. Using them on a real piece of work, and then defending the mark out loud, is what Rust, Price and O’Donovan found to work.

3. Put a strong answer beside a nearly-strong one. Keep going until your child can say precisely what makes the difference, and do it often enough that the language of the mark scheme stops being someone else’s vocabulary.

Our marking is built on the same idea. Every piece a student writes comes back marked against the examiner’s criteria, with the specific things that gained and lost marks named, so that over a term a student stops guessing what a marker wants and starts knowing.

In May your child will sit the exam without their teacher in the room, and a stranger will mark what they write. The teacher knew how your child thinks. The stranger will know only the page. So knowing exactly what the examiner rewards is the only help your child will have.

Sources