Refactoring-resistant plagiarism detection compares programs after normalization, not as text. Comments are stripped, identifiers are alpha-renamed consistently, literals get canonicalized, and the actual comparison runs over token n-grams, abstract syntax trees, or program dependence graphs. Rename n to value, collapse an if/else into a conditional expression, and a competent detector barely moves.
That property used to feel like a luxury. It doesn't anymore. A student with a browser tab open can ask a model to rewrite a function so it doesn't look copied, and get back something with different identifiers, different control flow, and identical semantics. The interesting question for anyone running a course or auditing a codebase is which classic detection techniques survive that, and which ones quietly stopped working while everyone was looking elsewhere.
What refactoring-resistant plagiarism detection actually compares
Every tool in this space starts the same way: run a lexer over the source and throw away everything a human would consider decoration. Whitespace, comments, formatters, and blank lines never reach the comparison stage. What's left is a token stream, and the design decisions start there.
Identifier normalization is where beginners get it wrong. The naive move is to replace every identifier with a single placeholder, so total, acc, and sum all become ID. That destroys information you need. Consider a - b and b - a: after naive replacement both become ID ID ID, and two genuinely different programs now look identical. The fix is alpha-renaming that respects binding structure, which is the same machinery a type checker uses to decide that two lambda terms are alpha-equivalent. Walk the token stream left to right, assign the first unseen identifier the symbol v0, the second v1, and so on, and reuse each symbol for every later occurrence of the same name. Now a - b becomes v0 - v1 and b - a becomes v0 - v1 only if the declarations were also swapped. The distinction survives.
Literal canonicalization is the smaller cousin. 3 * n + 1 and n * 3 + 1 should reduce to the same thing, as should 0x10 and 16. Some detectors go further and bucket all integer literals into a single class, on the theory that count = 5 and count = 17 are the same structural event. That's a tuning decision, and it trades a small amount of recall against a meaningful amount of precision.
The four families of similarity detection
Once normalization is done, the comparison itself falls into four families, each with a different failure mode.
| Family | Unit compared | Survives renaming | Survives reordering | Representative work |
|---|---|---|---|---|
| Text and line based | Characters, lines, LCS | No | No | Unified diff; most document-level checkers |
| Token fingerprints | Windows of k tokens | Yes | Partly | MOSS (Schleimer et al., 2003), JPlag (Prechelt et al., 2002) |
| Tree based | AST subtrees, tree edit distance | Yes | Yes | Baxter et al., 1998; GumTree (Falleri et al., 2014) |
| Graph based | Program dependence graphs, semantic birthmarks | Yes | Yes | Komondoor and Horwitz, 2001; Tamada et al., 2004 |
The ordering matters. Text-based comparison is cheap and useless against anyone who has ever pressed Tab in an IDE. Token fingerprints are the workhorse of academic detection and have been since the mid-2000s. Tree and graph methods cost more to run and are the only ones that hold up when a submission is rewritten rather than reformatted.
Winnowing, and the guarantee that makes MOSS predictable
MOSS, still running at Stanford and still the tool most professors have heard of, rests on the winnowing algorithm from Schleimer, Wilkerson, and Aiken's 2003 SIGMOD paper. The mechanism is small enough to describe in a paragraph.
Hash every contiguous window of k tokens. Then slide a window of size w over that sequence of hashes and, for each position of the sliding window, select the minimum hash. The set of selected hashes is the document fingerprint. The property that makes this useful is a guarantee: any shared substring of at least k + w - 1 tokens will produce at least one hash selected in both documents. Pick k = 15 and w = 4 and you have a promise that every common run of 18 tokens or more gets flagged, regardless of where it sits in the file. The parameters are a dial, not a mystery. Turn k down and you catch shorter fragments at the cost of noise.
JPlag takes a related approach and defaults its minimum match length to 9 tokens. That is a sane floor for Java coursework and far too low for a 20-line Python exercise, which is exactly the kind of thing that produces a wall of false positives on an intro assignment.
A worked example
Here are two submissions to the same problem, written to look as different as a determined student could make them.
def collatz_steps(n):
count = 0
while n != 1:
if n % 2 == 0:
n = n // 2
else:
n = 3 * n + 1
count += 1
return count
def cycle_length(seed):
steps = 0
value = seed
while value > 1:
value = value / 2 if value % 2 == 0 else 3 * value + 1
steps += 1
return steps
Run the families against each other and the scores diverge in a way that tells you something about each method. A line-level longest-common-subsequence score lands in the 35% range, which reads as clean. A token 20-gram comparison lands above 0.85, because after alpha-renaming the two token streams are nearly the same sequence. A strict subtree matcher drops to around 0.5, because the if/else collapsed into a conditional expression and the subtrees no longer align node for node. A tree edit distance that permits relabeling recovers to roughly 0.7.
Four detectors, four numbers, one pair of files. That divergence is the reason layering signals beats picking a favorite tool, and it's why a raw score without a visible diff is nearly useless as evidence.
When a model rewrites the algorithm, not the syntax
Everything above assumes the copier is doing cosmetic surgery. Large language models changed the economics of a different kind of edit. Paste in the function above and ask for an alternative implementation and you may get back something that replaces the parity test with a lookup, inverts the loop into a recursion, or tracks state in a dictionary instead of a counter. The token stream is now genuinely different. The AST is genuinely different. A tree edit distance of 0.2 here is honest: structurally, these are different programs.
What's left after the syntax has moved? Behavioral comparison, for one. Run both implementations over a few thousand inputs and diff the outputs or the execution traces. It's a real signal, and it's also weaker than it sounds, because any two correct solutions to collatz_steps produce identical outputs by construction. Behavioral equivalence is evidence that both programs work, not evidence that one was derived from the other.
The remaining signal is statistical, and it lives on the artifact itself. Perplexity and token-level surprisal measured against a reference corpus of student code tend to sit differently for model-written submissions than for human ones, and modern detectors can score a file on that basis. Our own tests suggest the signal is stronger on idiomatic-looking generated code and weaker on a three-line helper function where there simply isn't enough text to measure. Anyone who tells you an AI score is a verdict rather than a lead is selling something.
This is the gap that a AI code detector is meant to close, and it's the reason a peer similarity score alone stopped being sufficient around 2023. A submission can be 4% similar to every classmate and still be entirely machine-written. Two different questions, two different instruments.
Where refactoring-resistant detection falls apart
Three conditions produce most of the false positives I've seen in practice.
The first is shared scaffolding. If your assignment ships a 60-line skeleton and asks students to fill in 15 lines, every submission is 80% identical by construction and token n-grams will light up the entire class. The answer is to register the provided files as a base and exclude them from the comparison corpus. MOSS has done this for decades with its base-file flag; any tool worth using needs the same capability, and it needs to weight the score by the portion a student actually authored.
The second is small programs. A twelve-line submission that computes a running total has almost no degrees of freedom. for i in range(n): total += i is not a fingerprint of copying, it's the only reasonable way to write the loop. I once spent an afternoon chasing a cluster of 100% matches on a first-week exercise before finding an off-by-one in our window size that made short files look identical. The tool was right about the similarity and wrong about the meaning.
The third is assignments whose spec dictates the implementation. If your prompt says "use a stack, then pop until the queue is empty," structural similarity between submissions is noise. Students followed instructions.
Designing assignments where structural similarity means something
The pedagogical move is to leave room for divergence. Give the problem statement, the input format, and the complexity budget, then let students choose their data structures. On an assignment where a hash map and a sorted array are both valid, two structurally identical solutions are a much stronger signal than they would be under a step-by-step prompt. You've made structure informative by making it optional.
Tooling has to keep up with that. MOSS returns an HTML page of match URLs with no dashboard and no per-student trend. JPlag is a solid open-source token matcher with no web-source check. Dolos (Maertens et al., 2022) is the best of the newer open-source options and still compares peer submissions only. None of them will tell you that a submission also appears verbatim in a GitHub repository, or that a model probably wrote it.
A source code plagiarism checker built for the last three years needs to answer three questions on the same pass: how similar is this to the rest of the cohort after normalization, where on the open web does it appear, and how likely is it to be machine-generated. Codequiry runs token, AST, and fingerprint comparison across a cohort, matches against GitHub and web sources, and scores AI generation, then puts the results behind a review interface where you can read the two files side by side before you accuse anyone of anything. For engineering teams the same engine is reachable through a REST API and CLI, which matters if you're verifying contractor deliverables or checking a merge request. If you're weighing it against the tool your department has used since 2008, there's a fair breakdown of Codequiry vs MOSS worth reading.
The habit that matters more than any of it: treat similarity as an investigative lead, not a verdict. Open the diff. Read both submissions. A 0.91 from a normalized token comparison is a reason to look, and the look is what decides the case.
Frequently asked questions
Can renaming variables hide plagiarism in code?
No, not against a detector that normalizes identifiers before comparing. Alpha-renaming preserves the binding structure of the program, so n and value occupy the same slot in the normalized token stream. Renaming only defeats raw text or line-based comparison.
Does MOSS detect refactored code?
It detects reformatted, renamed, and reordered code, because winnowing runs over token k-grams rather than source text. It does not detect a rewrite that changes the algorithm, and it has no mechanism for checking web sources or AI generation.
What is a typical false positive rate for token-based detection?
It depends almost entirely on assignment design rather than on the tool. Short, tightly specified exercises with shared scaffolding can push false positives above 30% of the cohort. Open-ended problems where students choose their own approach drop that to a handful of pairs per hundred submissions. Excluding provided files is the single highest-leverage setting.
How do I tell AI-generated code from a human rewrite?
You usually can't from structure alone. Score the artifact statistically, check it against peer and web corpora, and interview the student about their own design decisions. A submission they can't explain is stronger evidence than any probability number.
If you're running a course this term, start by running one assignment through a code plagiarism checker that reports peer, web, and AI signals together, and see what your pass/fail threshold was actually letting through.