How to Detect AI Writing Safely with an AI Homework Marker
ChatGPT, Claude and similar tools are a browser tab away for most secondary students. They can help with learning and brainstorming, but they make one marking question harder to answer: how much of this assignment is the student's own work? A school marking platform with built-in detection is one way to approach that question, provided you understand what its output does and does not tell you.
The Trap of Absolute AI Detectors
When generative AI first reached classrooms, several companies responded by releasing absolute AI detectors. These tools promised certainty, often delivering definitive verdicts that a piece of text was "100% AI generated" or "100% human." However, relying on these binary, absolute detectors is fraught with severe pedagogical and ethical pitfalls. Text generation algorithms are complex, and the markers that differentiate human writing from machine text are constantly evolving.
The core issue with absolute detection lies in false positives and false negatives. A false positive occurs when a student’s hard work, perhaps structured exceptionally well or using advanced vocabulary, is incorrectly flagged as machine-written. The accusation of academic dishonesty can be devastating to a student's confidence and completely erode the trust between the learner and the teacher. Conversely, a false negative happens when a sophisticated prompt bypasses the detector entirely, rendering the tool effectively useless. In the context of a busy UK school, an AI homework marker must offer more nuance than a simple "guilty or innocent" verdict to truly support educational integrity.
A Smarter Approach: The Likelihood Score
Rather than dealing in absolutes, the most effective way to navigate the challenge of AI writing is through probabilistic detection. A sophisticated AI homework marker evaluates a text and assigns a likelihood score, acknowledging that identifying generated text is an exercise in probability, not certainty. GradeOrbit's school accounts, for example, return a 0-100 score that measures how far a new piece departs from the pupil's own earlier writing on file, with a Low, Medium or High confidence label and a plain-English explanation.
When an assessment returns a score of 85, the tool is not rendering a guilty verdict. Instead, it is signalling that the structural traits, vocabulary choices, and sentence patterns in the assignment differ markedly from how that pupil has written before. This probabilistic approach is fundamentally fairer. It shifts the tool from being an authoritarian judge to an investigative assistant, highlighting areas of concern while explicitly recognising the margin for error inherent in algorithmic detection.
Combining Technology with Professional Judgment
No matter how advanced an AI homework marker becomes, it cannot replace the essential component of effective teaching: professional judgment. Teachers hold critical contextual knowledge that an algorithm fundamentally lacks. You know your students. You understand their typical phrasing, their baseline literacy levels, and their individual academic journeys.
When an AI homework marker flags a high likelihood score, it should serve as a prompt for a constructive conversation rather than an immediate penalty. A teacher can compare the flagged assignment against the student's historic performance. If a Year 9 student who typically struggles with basic paragraph structure suddenly submits an exceptionally cohesive, postgraduate-level analysis of a Shakespearean sonnet, the teacher's professional judgment combined with the software's likelihood score provides a strong foundation to investigate further. The technology acts as a spotlight; the teacher serves as the seasoned interpreter.
Making the System Transparent for UK Schools
For an AI homework marker to be successfully integrated into a whole-school marking policy, transparency and flexibility are paramount. Educators must be transparent with students about the tools being used to evaluate their work and the criteria for investigating potential AI usage. This collaborative atmosphere fosters an environment of academic integrity rather than an adversarial game of cat and mouse.
Furthermore, predictable budgeting matters for whole-school adoption. GradeOrbit includes unlimited AI detection checks within its School teacher seats, rather than charging per check or asking departments to choose between tiers. School accounts operate under a signed Data Processing Agreement (DPA), so each assignment is compared against work that pupil has already had marked, providing a grounded comparison rather than an isolated score; a pupil with no earlier work on file is skipped rather than given a generic score. Individual teacher accounts do not include detection: student work there is redacted client-side and never stored. Submissions on every tier are never used to train AI models.
In Practice
A likelihood score is most useful when it starts a conversation rather than ends one: read it next to what you know of the student, compare the work with their earlier writing, and talk to them before drawing any conclusion. That keeps academic integrity checks fair without turning every flagged assignment into a dispute.
Detection on GradeOrbit school accounts works this way, comparing new work against the pupil's own earlier writing. A free school trial is available if your department wants to test it on real assignments.