AI marking, checked against senior examiners.
We marked exam board exemplar scripts with the same AI marking teachers use, then compared every mark with the one a senior examiner gave. Here are the results, how we measured them, and the limits.
Every AI mark compared with a senior examiner’s. The results, the method and the limits.
Measured 18 October 2026
Results by subject
Each AI mark compared with the senior examiner’s mark for the same question.
GCSE Maths
Pearson Edexcel · 54 scripts · 69 marks compared
Method and answer marks, 1–5 marks per question
- Within 1 mark
- 96%
- Within 2 marks
- 99%
- Within examiner tolerance
- 86%
- Average difference
- 0.2 marks
- Agreement score (QWK)
- 0.92
- Tendency
- In line with examiners
GCSE English
Eduqas · 73 scripts · 98 marks compared
Reading answers and level-marked writing and essays
- Within 1 mark
- 83%
- Within 2 marks
- 97%
- Within examiner tolerance
- 84%
- Average difference
- 0.7 marks
- Agreement score (QWK)
- 0.96
- Tendency
- In line with examiners
How much do examiners themselves vary?
No marker, human or AI, agrees with the senior examiner every time.
Maths is where examiners agree most. In Ofqual’s 2016–17 marking consistency study, GCSE Maths examiners’ paper totals differed from the senior examiner’s by a standard deviation of 1.9% of the marks available, the lowest of any subject studied. For GCSE English Literature the figure was 11.3%, around six times higher.
Extended answers leave room for honest differences of judgement. Across published studies of 6-mark answers, examiners gave exactly the senior examiner’s mark 46–75% of the time, and came within 1 mark 87–98% of the time.
Sources: Ofqual, Marking consistency studies: Summer 2016 and 2017 units (2018), Table 2; Ofqual, Marking consistency metrics: an update (2018), section 3.5. Both compare examiners with a senior examiner, the same comparison we use for the AI.
How we measured it
The same marking teachers get, scored against an answer key no one at GradeOrbit wrote.
The test
- Exam boards publish exemplar scripts: real candidates’ answers, marked by senior examiners against the board’s own mark scheme. The examiner’s mark for each question is our answer key.
- We removed every examiner mark and comment from the pages before the AI saw them.
- The AI marked each script through the same marking system teachers use, with the board’s official mark scheme and question paper. No extra notes, and nothing tuned on these scripts.
- Marking used Gemini 3.8 Flash, with Gemini 3.5 Flash as a backup when the first model is busy, as in normal use. 1.3% of scripts used the backup.
- Scripts: Pearson Edexcel GCSE Mathematics (1MA1), Higher and Foundation, June 2023; Eduqas GCSE English Language (C700, November 2022) and English Literature Shakespeare (C722, June 2022).
- In maths, 6 of 60 scripts were left out because our system could not read the mark scheme’s mark points for two questions.
What the numbers mean
- Within 1 mark / within 2 marks
- How often the AI’s mark for a question was no more than 1 (or 2) marks away from the senior examiner’s.
- Within examiner tolerance
- For questions marked point by point, the exact mark. For level-marked answers, within 10% of the marks available (2 marks on a 20-mark essay), the kind of leeway exam boards allow between examiners.
- Average difference
- The average gap between the AI’s mark and the examiner’s, in marks, whichever direction it fell.
- Agreement score (QWK)
- Quadratic weighted kappa across all questions: 1 is perfect agreement, 0 is no better than chance. It counts big misses more heavily than small ones.
Figures above 99% are shown as 99%.
The limits, stated plainly
What this study can and cannot tell you.
The limits
- Exemplar scripts are chosen by the exam board to show each mark level, so they are not a random sample of real classes.
- The scripts are published online, so the AI may have seen some of them during its training.
- We ran this study ourselves. It has not been independently reviewed.
- The samples are small: 69 maths marks and 98 English marks.
- Results are for the two subjects and boards shown above. Other subjects and boards may differ.
More subjects, and you
We have also tested Edexcel GCSE English Language, Eduqas GCSE Geography, AQA GCSE History and AQA A-level Psychology, and we are finalising the figures for those. We will add each subject here when its results are ready.
The AI suggests every mark. You review it, change it if you disagree, and approve it. The final judgement is always yours.
See how it marks your students.
Ten free credits to start — enough to mark ten students. No card, no catch.
1 credit per student · you approve every mark