The Danger of Overconfidence

In the week after results day, the GCSE parents’ groups on Facebook fill up with the same message, posted again and again by families who have never met. Here, word for word, is one of them.

“My daughter had consistently been graded as 9 throughout in English Language AQA. She was graded as a 6 – 1 mark off a 7. We sent off paper 2 to be reviewed as it was significantly lower than paper 1. We’ve just had the review back: unchanged! She is devastated as she cannot see what she did wrong.”

Suppose she were your daughter. She has been told for a year that she writes at grade 9, the paper comes back a 6, and the review says the 6 is right. Where would you look first? Most of us would look at the exam itself, for a hard question, a bad day or a harsh marker. The review has already ruled out the marker, and a bad day usually leaves a family knowing exactly what went wrong. The answer is more likely to be in the year before the exam than on the morning of it.

Feeling sure is not knowing

Start with something simpler than an exam. How well do you understand a zip? Most people would say very well, since they open one every morning without giving it a thought.

In 2002 two psychologists, Leonid Rozenblit and Frank Keil, asked people that question about a few ordinary objects, such as a zip, a lock and a flush toilet, and most of them rated their understanding highly. Then the psychologists asked for something harder. Each person had to write down, step by step, how the object actually works.

Try it now with the zip. What holds the two rows of teeth together, and what does the little slider do to them as it moves? Most people manage a sentence, perhaps two, and then the explanation runs out.

That is what happened in the experiment. When people rated their understanding again, after trying to write it down, their scores went down. The only thing that had changed was that somebody had asked them to explain. Rozenblit and Keil called this the illusion of explanatory depth.^[1]

Does the same thing happen with everything we know? It does not. With stories, and with procedures that are done one step after another, the researchers found no drop at all. With simple facts the drop was much smaller. It was explanations, the answers to how and why, that fooled people most.

We think this is why good students are not safe from it. Your child probably knows the plot of An Inspector Calls well and could give you a dozen quotations from memory. Facts like these are easy to check from the inside, because either the inspector’s name comes to mind or it does not. An explanation is much harder to check. A student can sit through a whole lesson on why Priestley puts an inspector in that dining room, follow every word, nod in the right places and walk out certain, because following an explanation feels almost exactly like being able to give one. The two only come apart when somebody asks for the explanation itself, and an English exam asks for very little else.

So go back to the year before the exam. When a year of top marks ends in a grade that nobody can explain, the reason we see most often is that the student was told their work was good, but nobody showed them what made it good. In the exam hall, with the clock running and no teacher to glance at, a paragraph worth a 6 and a paragraph worth a 9 felt exactly the same to write.

The obvious fix is more revision: read the book again, read the notes again. In 2006 two other psychologists, Henry Roediger and Jeffrey Karpicke, compared students who studied the same material again with students who tested themselves on it instead. A few minutes later, the students who had re-read remembered more. On the later tests, the students who had tested themselves remembered substantially more, even though the extra reading had made the other group more confident.^[2]

Three questions, four minutes, tonight

Rozenblit and Keil broke the illusion with a single move, which was to ask for the mechanism instead of the fact: not what the thing is called, but how it works and why. You can make the same move over dinner without knowing any English Literature yourself, because the answers are written underneath each question.

1. “What is AO1, and what does the top band actually ask for?”

Ask both halves, and listen for the distance between the two answers.

Nearly every student can answer the first half, because it is printed at the top of the page. AQA’s own words for AO1 in GCSE English Literature are “Read, understand and respond to texts.” So your child will probably tell you that it means understanding, and that you have to understand the book.

Now turn to the page the examiner actually marks from. At the top band, Level 6, which AQA calls “Convincing, critical analysis and exploration”, the same AO1 asks for a “critical, exploratory, conceptualised response to task and whole text”, with “judicious use of precise references to support interpretation(s)”.

Put the two descriptions side by side and they turn out to be asking for different things. “Understand the text” is something a student can believe they have done the moment they close the book. A conceptualised response to the whole text is one argument about what the whole book is doing, carried through every paragraph of an essay, and no amount of re-reading will take a student from the first to the second.

So if your child’s answer is “understanding”, they have given you the summary printed at the top of the page, which is not what anybody is marked against. We have written about that gap on its own in Knowing the mark scheme is not understanding it.

Here AQA’s mark scheme is the example. On the other boards we checked, the same gap sits between the summary of AO1 and the top level.

Edexcel and OCR print the same AO1 as AQA, “Read, understand and respond to texts”, in their own GCSE mark schemes.^[3][4] This is because the government sets that wording for GCSE English Literature.^[5] Each board then words its top level in its own way:

Edexcel GCSE, Level 5: “A critical style is developed with maturity, perceptive understanding and interpretation”.^[3]

Eduqas GCSE, Band 5: students “use a sensitive and evaluative approach to the task and analyse the extract and wider text critically”.^[6]

OCR GCSE, Level 6: “Coherent critical style sustained in an informed personal response to the text, showing consistently perceptive understanding”.^[4]

Edexcel International GCSE uses different words for AO1: “Demonstrate a close knowledge and understanding of texts, maintaining a critical style and presenting an informed personal engagement”. Its top level, Level 5, asks for “assured personal engagement and a perceptive critical style”.^[7]

Cambridge IGCSE also uses different words for AO1: “show detailed knowledge of the content of literary texts, supported by reference to the text”. Its top band, Band 8, asks for that knowledge to be shown “by incorporating well-selected reference to the text skilfully and with flair”.^[8]

We have not compared the CCEA mark scheme.

2. “Why does Shelley use the sonnet form in Ozymandias?”

A sonnet is fourteen lines with a turn somewhere in the middle, a shape built to make an argument feel proven. It is also the shape of a love letter, the form people reach for when they want to lift somebody up. Shelley points it at a broken statue in a desert, the statue of a king whose empire has turned to sand. So the form that usually flatters is being used to take a man apart, and the shape of the poem becomes part of the insult.

3. “Why does Tennyson use the ballad form in The Charge of the Light Brigade?”

A ballad is old technology, with its regular beat, its rhyme and its repetition, built so that a story could be carried in people’s heads and passed on, long before most people could write. Six hundred men rode at the wrong guns because somebody gave the wrong order. Tennyson set that down in the one form designed to stop a story from being forgotten, so the poem is doing more than describing the charge: it is keeping the memory of it alive.

Listen for the link, not the label

You are not marking your child, so you do not need to know the right answer yourself. You are listening for one thing only, which is whether the answer joins the choice to its effect.

A blank tends to sound like “It’s just the form he used,” or a shrug, or the plot of the poem retold at length to fill the silence. A real answer sounds more like this: “It’s a sonnet, and a sonnet builds up to prove something, so using one on a smashed statue is basically a joke at the king’s expense.” Clumsy and unfinished as it is, that answer is still a grade 8 or 9 thought, because it has hold of the machinery rather than just the label.

If all you get is blanks, nothing has gone wrong and nobody is behind. Your child has the same illusion nearly everybody has, and the lucky part is that you have found it at the kitchen table, with time still left to do something about it.

The lesson is explaining, not poetry

A student who is never asked to explain anything can walk into the exam hall certain and walk out unable to say what happened. Most of these students worked hard and were taught well enough. What they missed was someone asking them to say it out loud while there was still time to be wrong.

That is what a marked essay is for. A number at the top of the page cannot do it, because a 6 tells a family where the work stopped but never which sentence stopped it. What helps is a line drawn under one particular sentence, with a note in the margin naming the part of the thinking that went missing.

Four things to do this week

1. Ask one of the three questions at dinner, then stop talking. Leave the silence alone until your child fills it, because the pause is where sounding sure and being sure come apart.

2. Swap the fact question for the mechanism question. Instead of “What happens in Act Two?”, try “Why does Priestley put the inspector in that room?” A revision guide can answer the first question for them. Only your child can answer the second.

3. Read the top band out loud together. Search for your child’s exam board and subject with the words “mark scheme”, find the highest level, and read out the sentence the examiner actually marks from. It takes about ten minutes, and you only need to do it once.

4. Ask the same three questions again in a fortnight. Whatever your child can tell you then, with no book open and nobody prompting, is the part they genuinely own.

If you would like to see that kind of marking on something your own child has written, we will mark one piece, free.

Sources