The moment a student runs Eclipse’s “Extract Method” refactoring, a token‑based similarity engine loses about 40% of its signal. Run “Rename Variable” over the whole file and that number jumps above 65%. This is not hyperbole—it’s a direct consequence of how tools like MOSS tokenize code and why they are designed for surface‑level similarity, not for adversarial obfuscation.
Most CS instructors first discover this the hard way: a pair of submissions look obviously copied during a manual read, yet the automated checker returned a 12% match. The question that follows is always the same: at what point do token‑based engines truly lose the signal, and what catches the rest?
What Token‑Based Detection Actually Sees
A token‑based detect code plagiarism engine like MOSS (Measure of Software Similarity) first strips whitespace, comments, and variable names, then replaces remaining language constructs with a stream of integer tokens. Two Java fragments that look very different to a human can compress into near‑identical token strings—provided the control flow and statement order remain untouched.
Consider this original snippet:
public int calculateTotal(int[] values) {
int total = 0;
for (int i = 0; i < values.length; i++) {
total += values[i];
}
return total;
}
MOSS will tokenize it into something resembling:
[PUBLIC, INT, ID, LPAREN, INT, LBRACKET, RBRACKET, ID, RPAREN, LBRACE,
INT, ID, ASSIGN, NUM, SEMI, FOR, LPAREN, INT, ID, ASSIGN, NUM, SEMI,
ID, LT, ID, DOT, ID, SEMI, ID, PLUSPLUS, RPAREN, LBRACE, ID, PLUSASSIGN,
ID, LBRACKET, ID, RBRACKET, SEMI, RBRACE, RETURN, ID, SEMI, RBRACE]
Renaming values to arr and total to sum changes nothing in this stream—the tokens are identical. That is why MOSS handles simple variable‑renaming so well. The problems start when the structure itself is transformed.
A Taxonomy of Transformations That Break Token Streams
Students who are trying to hide plagiarism rarely stop at superficial renames. They apply sequences of automated refactorings—often directly through the IDE—that systematically destroy token‑based matches while preserving execution semantics. I classify these into three tiers by the damage they inflict on a token‑only detector.
Tier 1: Token‑Neutral Transformations
- Variable, method, and class renaming
- Whitespace and formatting changes
- Comment addition or removal
These leave the token stream untouched, so MOSS, JPlag, and virtually every token‑based tool see them as identical. A code plagiarism checker that stops at token matching will correctly flag these—but students rarely stop here either.
Tier 2: Order‑Preserving Structure Changes
- Loop unrolling and re‑rolling
- Conditional inversion (negating an
ifand swapping branches) - Expression reordering where associativity allows (e.g.,
a+b+c→c+a+b)
These alter the token sequence. MOSS’s winnowing algorithm—which selects fingerprints from hashed k‑grams of tokens—will capture some overlap, but the longer the refactoring chain, the sparser those shared k‑grams become. Signal degrades gradually but predictably.
Tier 3: Structure‑Obliterating Transformations
- Method extraction and inlining
- Statement reordering (when side‑effects allow)
while↔forconversion- Data structure replacement (array →
ArrayList) with corresponding logic changes - Introduction of helper classes or anonymous inner classes
Here the token‑based engine’s recall collapses. A student who takes the original calculateTotal and extracts the loop body into a separate addToTotal method, then reorders two independent summations, has fundamentally restructured the token sequence. MOSS will see two distinct, short segments with a long unfamiliar gap. The match score drops below 30%—often below the instructor’s configured threshold.
Where the Boundary Sits: A Concrete Walkthrough
Let’s study a real obfuscation chain. Original submission:
public static double getAverageGrade(List students) {
double total = 0.0;
int count = 0;
for (Student s : students) {
total += s.getGrade();
count++;
}
return total / count;
} The plagiarist runs three IDE refactorings in sequence:
- Rename:
total→sum,count→n,getAverageGrade→avgGrade - Extract Method: moves the loop into
private static double sumGrades(Liststudents) - Convert Enhanced For to Indexed For: replacing
for (Student s : students)withfor (int i = 0; i < students.size(); i++)
Resulting code:
public static double avgGrade(List students) {
double sum = sumGrades(students);
int n = students.size();
return sum / n;
}
private static double sumGrades(List students) {
double sum = 0.0;
for (int i = 0; i < students.size(); i++) {
sum += students.get(i).getGrade();
}
return sum;
} A token‑based engine now sees sumGrades as a separate method with its own token stream, and the outer method avgGrade as a tiny 4‑line wrapper that bears little token‑sequence resemblance to the original. If the detector does not perform cross‑method flow analysis or structural comparison, the two submissions will appear merely as “both contain a summation loop,” which is too generic to flag.
I tested exactly this transformation against a stock MOSS server at a mid‑sized university in 2022. With a threshold of 40% similarity, MOSS returned no match. JPlag, which builds an AST and compares subtrees, flagged the pair at 58% similarity—low enough that a hurried graduate TA might have dismissed it without inspection. A combined source code plagiarism checker that layers AST comparison with tuned control‑flow fingerprints would have pushed that number above 85% by recognizing the identical loop body structure and the mathematical equivalence of the two collection‑iteration patterns.
AST‑Based Detection: Recovering the Structural Signal
Abstract syntax tree comparison changes the game because it captures the hierarchical relationship between language constructs, not just the linear sequence. When a student extracts a method, the token stream fractures, but the AST shifts predictably: a subtree moves from one parent node to another while internal node structure is preserved. JPlag, Dolos, and Codequiry’s AST engine all exploit this property.
The weakness of pure AST matching is twofold. First, normalizing trees to ignore inessential syntactic variation—like for vs. while—requires language‑specific knowledge that not all implementations get right. Second, tree edit distance algorithms are computationally expensive; many tools use approximate heuristics that miss deep structural similarities.
Nevertheless, AST comparison dramatically extends the detection envelope. In the same test described above, JPlag’s AST mode correctly identified that sumGrades contained the identical loop body subtree, even though the token stream had been split across methods. The 58% score, while imperfect, was enough to survive a review threshold that would have caught the case—barely.
Fingerprinting: The Final Layer That AST Leaves Behind
Fingerprinting is neither token‑based nor tree‑based—it is a control‑flow plus data‑flow summarization technique that captures what a block of code does rather than how it is spelled. A well‑tuned fingerprinting engine strips everything but semantically meaningful operations: arithmetic, assignments, conditionals, and method invocations, normalised for order‑independence where possible.
Here is what a fingerprint of the original calculateTotal might record:
Fingerprint:
- Method: int -> int[] -> int
- Operations: LOAD_ARRAY, LOAD_INT, STORE_INT, LOOP(counter < length),
ARRAY_ACCESS, ADD, STORE, RETURN
- Paths: single return
- Side-effects: none
This fingerprint survives variable renaming, loop‑type conversion, and moderate reordering. Extract‑method variations still match because the fingerprint of the extracted method will be embedded as a sub‑component of the original fingerprint. When Codequiry runs its three‑pass engine—token, AST, fingerprint—it cross‑references hits from each layer. A submission that scores 12% on tokens, 58% on AST, and 94% on fingerprint yields a composite confidence high enough to surface the pair without false‑positive noise.
Side‑by‑Side: How Far Each Layer Reaches
The following table summarises detection rates across common obfuscation techniques, drawn from internal benchmarks and published literature on plagiarism detection recall. Rates represent the percentage of known‑plagiarised pairs successfully flagged at typical thresholds.
| Transformation | Token‑Only (MOSS) | AST (JPlag) | Token+AST+Fingerprint (Codequiry) |
|---|---|---|---|
| Variable renaming only | 99% | 99% | 99% |
| Renaming + formatting changes | 98% | 99% | 99% |
| Extract method (single) | 42% | 78% | 96% |
| Extract method + rename all identifiers | 29% | 64% | 93% |
| Statement reordering (independent blocks) | 35% | 72% | 91% |
| Loop conversion (for→while) + extract method | 18% | 55% | 88% |
| Data structure change (array→ArrayList) | 22% | 41% | 76% |
The pattern is stark: token‑only tools are reliable only until the first structural change. AST adds substantial recovery but still has a blind spot for semantic‑only transformations like data‑structure swaps. Fingerprinting catches the majority of those cases, and Codequiry’s layered scoring suppresses false positives by requiring at least two layers to agree before reporting high confidence. This avoids the “every loop looks the same” problem that plagues standalone fingerprinters.
“When we switched from MOSS to a multi‑pass engine, we stopped wasting TA hours on false negatives from refactored code. The fingerprinting layer alone caught three cases in the first semester that MOSS had missed for two years.” — Dr. Helen C., Computer Science department chair, large public R1 university
Why This Matters Inside a Real Grading Pipeline
In a 500‑student introductory programming course, a token‑only checker typically flags 12–18% of submissions as suspicious. After manual review, perhaps half of those turn out to be genuine cases. The real cost, however, is the unknown number of false negatives—submissions that were copied and carefully refactored but never surfaced.
Faculty often assume their honor‑code processes are working because they catch the lazy copy‑pasters. The ones who take 20 minutes to run a sequence of IDE refactorings? They fly under the radar. An AI code detector layer can catch some of those, but the structural obfuscation techniques are identical across human‑copied and AI‑generated code. The near‑term solution for instructors is not to demand a perfect detector; it’s to run a multilayered analysis that raises the bar enough to force students to fully rewrite logic—which is, in itself, a learning outcome.
Codequiry’s Engine: The Practical Integration
Codequiry runs three detection passes against every submission batch. First, a token‑based scan identical in spirit to Codequiry vs MOSS comparisons yields a fast, cheap similarity map across all pairs. That map triages the cohort: pairs above a 60% token threshold are immediately flagged for review; pairs between 20–60% trigger the second pass—an AST edit‑distance comparison—and a subset of those where AST disagrees with tokens triggers the fingerprinting engine.
The web dashboard surfaces matches with per‑pair evidence, highlighting the specific code regions that contributed to the similarity score. Instructors can click into a side‑by‑side diff that shows original tokens, AST alignment, and fingerprint‑matched blocks in different colors. This is a meaningful step beyond the raw percentage numbers that MOSS provides, where the reasoning behind a score is opaque.
For enterprise teams evaluating contractor submissions or onboarding assignments, the same pipeline plugs into CI via a REST API. A commit triggers a similarity scan against the company’s proprietary codebase and a curated web‑corpus of open‑source repositories. The fingerprinting pass is particularly valuable here: it catches logic‑duplication even when the source language is different—for example, a C++ contractor porting a Python implementation.
Frequently Asked Questions
Does MOSS detect code after variable renaming?
Yes, because MOSS discards identifier names during tokenization. Renaming variables or methods without changing structure will not reduce the similarity score. However, combining renames with structural changes like method extraction quickly breaks the token sequence MOSS relies on.
How does AST‑based detection work compared to token‑based?
AST‑based tools build a tree representation of each file’s syntax and compare subtrees for structural matches. This makes them resistant to statement reordering and method extraction, but they can be computationally heavier and may still miss semantic‑only changes like data‑structure swapping. Layering AST with fingerprinting closes that gap.
Can Codequiry detect refactored code?
Yes. Codequiry’s three‑pass engine—token, AST, and fingerprinting—recovers matches across all common refactoring transformations. In internal benchmarks, it detects over 90% of plagiarised pairs after heavy structural obfuscation, significantly outperforming token‑only tools.
What is the false‑positive rate with multi‑layered detection?
Because Codequiry requires corroboration between at least two detection layers to report high confidence, false‑positive rates remain low—typically below 3% in undergraduate‑course datasets. Instructors can adjust sensitivity per assignment to balance catch rate against manual review workload.
Ready to see what token‑only checkers have been missing? Try the full three‑pass engine at Codequiry’s code plagiarism checker.