In the spring of 2021, I sat with a student who could not explain a regular expression she had submitted the night before. It was not copied from a classmate; JPlag found nothing above a 12% match. It was, she eventually admitted, generated by an early version of GitHub Copilot. That moment pushed me into the history of AI-generated code detection, a phrase I had barely used before that semester. The history is shorter than most people expect, but it builds directly on three decades of code similarity work.
The MOSS Era Solved Copying, Not Authorship
When I started teaching introductory Java in 2008, our plagiarism workflow was simple. We ran MOSS, the Stanford tool that Alex Aiken and colleagues built in 1994. MOSS tokenized source files, stripped whitespace and comments, and selected a subset of k-gram fingerprints using the 2003 winnowing algorithm from Schleimer, Wilkerson, and Aiken. JPlag, which Guido Malpohl and Lutz Prechelt developed at Karlsruhe in the late 1990s, used a related token-based approach and gave us side-by-side comparisons. Both tools answered one question well: did this student copy from another student?
The underlying idea was elegant. Tokens, not raw characters, survived renaming. Fingerprinting plus winnowing made large cohorts tractable. A simplified version of the preprocessing looked roughly like this.
# Simplified MOSS-style preprocessing for Java, circa 2003.
# Strip whitespace, tokenize, hash k-grams, keep fingerprints.
tokens = "public static void sort(int[] a)".split()
k = 5
fingerprints = [hash(tuple(tokens[i:i+k])) for i in range(len(tokens) - k + 1)]
That approach caught copy-paste, light reformatting, and some variable renaming. It did not catch original work produced by a model. The reason matters. A token fingerprint finds shared structure between submissions; it assumes the copied artifact exists somewhere else in the corpus. AI-generated code often has no peer twin. That was the core blind spot we carried into 2021.
Any modern code plagiarism checker still owes a debt to that token fingerprint design. The old tools are not obsolete. They are just insufficient for the new problem.
Code Stylometry Arrived Before LLM Detectors
Before ChatGPT, researchers were already asking whether code contains authorial fingerprints. In 2015, Aylin Caliskan-Islam and colleagues published work on de-anonymizing programmers via code stylometry. They showed that abstract syntax trees, control flow, and lexical choices could identify a programmer with surprising accuracy, even across compiled binaries in some experiments. That lineage matters because it shifted the question from was this copied to does this look like the person who submitted it.
Code stylometry gave us features we still use today. Identifier entropy measures how varied and specific variable names are. Comment density and placement reveal habitual behavior. Whitespace regularity, line length, and brace style carry signal. These methods were not built for AI detection, but they trained a generation of researchers to treat code as a stylistic object, not just a functional one.
I remember a graduate student in 2016 trying to use stylometry to flag purchased assignments. The idea failed in practice because students code so differently under stress, under autograders, and under time pressure. That failure taught us an early lesson: style is real, but it is noisy, and a single stylistic feature cannot carry an academic misconduct charge.
Why ChatGPT Altered the Detection Problem
Copilot entered technical preview in June 2021, but most of my colleagues did not see a flood until ChatGPT opened to the public in November 2022. The submissions changed almost overnight. Variable names became bland. Comments were surprisingly complete. Edge cases were explained with a patience that first-year students rarely display. None of it matched a classmate, and most of it did not match Stack Overflow either.
The old similarity assumption broke. A student could ask a model for a recursive binary search, add their own variable names, and produce zero peer matches. What remained was a different kind of trace: a statistical regularity that human code under deadline pressure rarely has.
We started comparing suspicious files to known generated samples. The difference was not in any single line. It was in the distribution. That observation is what current detection tools operationalize.
How AI-Generated Code Detection Works Under the Hood
Modern detectors generally belong to two families. The first family borrows from text detection. Perplexity asks how predictable a token sequence is under a language model. A low-perplexity sequence means the model assigns high probability to most tokens. Generated code often sits in that high-probability groove. Burstiness, also borrowed from text analysis, looks at whether complexity spikes irregularly. Human code has uneven bursts; model-produced code tends toward smoothness.
The second family trains classifiers on labeled human and AI-generated code. These classifiers use features like token frequency, whitespace patterns, comment density, AST depth, identifier entropy, and even the distribution of line lengths. A classifier might learn that a file with unusually regular indentation, low identifier entropy, and evenly spaced explanatory comments is more likely generated. None of these features is decisive alone. The detector aggregates them into a probability.
# Perplexity as a generation signal, adapted to code tokens in 2023.
# Low values often mean the token sequence was highly predictable to a model.
import math
def token_sequence_perplexity(log_probs):
return math.exp(-sum(log_probs) / len(log_probs))
For a classroom, the useful output is not a binary AI or human label. It is a per-file probability together with the evidence that produced it. The AI code detector I currently use reports per-file probabilities and a cohort distribution, not a single pass/fail score. That matters when I have to stand in front of an academic integrity committee.
Because manual review does not scale, I now rely on a dedicated AI code detector that reports per-file probabilities instead of a single course-level score. That design choice matters. A class-level average hides the one file in an otherwise human project that was generated wholesale.
Where Statistical Detectors Still Stumble
I want to be honest about the limits. These tools are not oracles. One tool we tried in January 2023 returned a 98% AI score for a student's hand-written assembly file because it was short, repetitive, and nearly comment-free. We have not let a single numeric score drive a sanction since. Short submissions, boilerplate-heavy files, lab templates, and students who write in a terse textbook style can all trip these detectors.
False negatives are just as real. A student can ask the model to insert spelling errors, rewrite comments, or vary whitespace. That reduces the signal but does not remove all of it. We have not tested our own threshold choices past a few hundred Python and Java submissions, so I treat every score as a prompt to look more closely, not as a verdict.
The uncomfortable lesson of the past three years is that originality cannot be reduced to a single numeric score.
The academic line has also moved. I now care less about a binary human-versus-machine label and more about what a student can explain. A student who used Copilot to scaffold a loop and then debugged it for an hour has a different case from a student who pasted an entire solution and cannot discuss a single design choice. The detection tool should open that conversation; it should not replace it.
The Case for Stacking Signals in One Workflow
MOSS and JPlag still do what they were built for. They are fast, interpretable, and free or close to free. Dolos put a modern interface on the same similarity idea. Turnitin added an AI writing indicator, but source code is not an essay. What none of them do well is combine peer similarity, web source matching, and model-based AI probability in one report.
Codequiry's approach is to run those three checks together. Peer similarity catches copied and lightly refactored code. Web matching catches Stack Overflow and GitHub snippets. AI detection catches synthesized code that has no copy source. The stacked report has held up better in our grading meetings than any single score, because each signal catches cases the others miss.
A submission that scores 91% similar to a classmate needs no AI detector. A submission that scores 88% likely AI with no peer match needs a different kind of follow-up. The combination tells you which conversation to have before office hours even begin.
Three decades ago, we learned that code copying leaves fingerprints. Two years ago, we learned that generated code leaves different fingerprints. The next step is not a single perfect detector. It is treating AI probability as one input among peer and source matches, then walking into office hours with specific questions. If you are setting up a workflow that catches both copied and generated code, the most practical place for me to point you is Codequiry's code plagiarism checker, which now runs peer, web, and AI checks in one submission.