Judgment

Can teachers tell if you used AI? What detectors actually do

By the Keep Your Head desk · About a 6-minute read

Two people need this answer for opposite reasons: a student wondering if they'll get caught, and a student who wrote every word themselves and just got accused anyway. The honest answer serves both, and it's uncomfortable for the schools in the middle: these tools do not work well enough to decide anything on their own, and the people selling them mostly say so in the fine print.

What OpenAI found when it tried to build one

The strongest evidence doesn't come from critics — it comes from the company with the most to gain from a working detector. OpenAI built an AI text classifier, then shut it down, and its guidance for educators explains why in unusually plain language: "Our research into detectors didn't show them to be reliable enough given that educators could be making judgments about students with potentially lasting consequences."

The specific failure is the one that matters. In OpenAI's words, one key finding was that these tools "sometimes suggest that human-written content was generated by AI" — and when they trained their own, it labeled human-written text including Shakespeare and the Declaration of Independence as AI-generated.

Read that again, because it's the whole article. The documents being flagged as machine-written were written centuries before the machine existed.

Why the errors aren't random

If detectors were wrong evenly across everybody, they'd be useless but at least fair. They aren't wrong evenly. The pattern researchers keep finding is that detectors flag writing that is simple, even, and grammatically tidy — because that's what predictable text looks like statistically, and predictable is the only thing these tools can actually measure.

Which writers produce simple, even, tidy prose? Non-native English speakers, who often write in shorter and more conventional constructions. Students who were taught rigid five-paragraph structure. Autistic writers. Anyone who edits carefully. Anyone who uses a grammar checker. The tool isn't detecting AI; it's detecting an absence of idiosyncrasy, and then a school treats that as evidence of cheating.

This is the same failure described in how to check if an AI answer is actually true, wearing a different hat: a confident-sounding output with no visible uncertainty, being trusted because it arrived with a number attached. A detector saying "87% AI" is not a measurement. It's a guess with a decimal point on it.

The asymmetry nobody accounts for

Here's what makes this worse than an ordinary accuracy problem. The two ways a detector can be wrong do not cost the same.

A false negative — a student used AI and the tool missed it — costs essentially nothing. One assignment slips through. Nobody's life changes.

A false positive — a student wrote it themselves and the tool says otherwise — asks an eighteen-year-old to prove a negative about their own mind, in front of someone with power over their transcript. There is no clean way to prove you thought of something. Even when the case is dropped, the student has learned that being careful and articulate is now a risk.

A tool whose two error types are that lopsided should require overwhelming accuracy before anyone acts on it. Detectors aren't close.

If you've been accused and you didn't do it

Being right doesn't automatically help you here, so be methodical rather than indignant:

• Ask what the actual evidence is. If the answer is "the detector said so," that's a score, not evidence. Ask what the tool's own documentation says about false positives — most vendors disclose them.
• Produce your process, not your innocence. Version history in Google Docs or Word, drafts, notes, browser history, timestamps. This is the single most effective thing you can show, which is also why it's worth writing in a tool that keeps history before you ever need it.
• Offer to talk about the content. Someone who wrote a paper can discuss why they cut a paragraph. It's a far better test than any classifier, and most teachers know it.
• Ask for the policy in writing. Many institutions have no formal rule about detector evidence. That gap tends to matter if the situation escalates.

If you did use AI — the actually useful question

Not "will I get caught," which is a coin flip on a broken tool. The better question is what you're trading away. Using AI to explain a concept you're stuck on, to argue against your thesis, or to check your own reasoning makes you better at the subject. Having it produce the paragraphs means you didn't practice the thing the assignment existed to make you practice — and the gap shows up later, in a room where you can't paste the question anywhere.

If you want the tool without the trade, make it interrogate you instead of write for you:

I'm writing an essay arguing [your position].
Do not write any of it for me.

Instead: ask me the five questions a sceptical reader
would ask, one at a time, and wait for my answer each
time. Then tell me which of my answers was weakest
and why.

You still write every word. You just write a better version, because something pushed back before your teacher did.

Keep your head:

---

Get one of these a week. Our free newsletter sends one genuinely useful AI habit and one judgment check every week — no hype, four-minute read. Subscribe on the home page.

Related: Should you let your kid use AI for homework? and How to check if an AI answer is actually true.