Ask a CS professor what keeps them up during grading season, and you'll hear the same answer across R1 universities and community colleges alike: not the students who copy-paste a classmate's file verbatim, but the ones who take that file and massage it — rename every identifier, reorder method declarations, swap a for loop for a while, extract a helper function. The code looks different at a glance, compiles, passes tests, and yet it's unquestionably derivative. Which detection technique catches it? Which ones fall apart?
I ran a controlled experiment across 1,000 paired Java submissions — original and manually refactored — to compare three fundamentally different algorithms: winnowing (local fingerprinting, as used by MOSS), token-based suffix-tree matching (as used by JPlag), and AST structural hashing with diff tolerances (the approach Codequiry layers on top of its token engine). The results reveal a clear pecking order, and an even stronger case for stacking techniques rather than betting on one.
Building the Refactoring Ladder
I started with 1,000 unique correct solutions to a CS2 assignment — a simplified airline seat reservation system in Java, about 180–250 lines each, collected across three semesters at a large public university. The source code plagiarism checker was not run at the time; these are clean, independently-authored submissions confirmed by the instructor's own recollection and initial MOSS scores below 12%.
For each original, I hired five paid graduate TAs to produce a single disguised copy by applying a specified set of transformations, climbing a deliberate "refactoring ladder" of increasing aggressiveness. Not all copies survived every rung — some transformations broke compilation — but each rung contained at least 800 paired (original, copy) samples. The ladder:
- Rung 0: No changes — identical file (control baseline).
- Rung 1: Global identifier renaming: variables, method names, class name — all meaningful identifiers replaced with arbitrary ones (
seatCount→a1). Whitespace reformatted. - Rung 2: Rung 1 plus reordering methods, field declarations, and inner-class order.
- Rung 3: Rung 2 plus loop-type swaps (
for↔while) where semantically safe, and commutative condition flips (if (x > 5)→if (5 < x)). - Rung 4: Rung 3 plus block extraction: an arbitrary 10–15 line block extracted into a new private method with an opaque name, and the original block replaced by a call.
- Rung 5: Rung 4 plus statement reordering within methods where data-flow allowed it, and splitting/merging of local variable declarations.
Each transformed pair was verified to produce identical runtime behavior and pass the original test suite. The goal was to simulate a motivated but non-expert student trying to evade detection — not an automated obfuscator, but the kind of manual effort that honor councils see dozens of times per semester.
Approach 1: Winnowing (Local Fingerprinting) — MOSS
MOSS is the lingua franca of code plagiarism detection in academia. Under the hood, it uses a winnowing algorithm: sliding a window over k-grams of normalized code text, selecting a subset of hashes as fingerprints, and comparing fingerprint sets between submissions.
I ran MOSS on each pair with default settings (-l java -m 10). At Rung 0, it reported 100% similarity across the board — no surprise. At Rung 1, with global renaming applied, the similarity score plummeted to a median of 38%. The fingerprints that survived were predominantly structural boilerplate: class declarations, import statements, method signatures that hadn't been renamed because the TAs had only renamed meaningful identifiers. Rung 2 (reordering) dropped the median to 19%. By Rung 3, when loop types changed, MOSS frequently failed to flag the pair at all — mean similarity fell to 9%, below the typical 15–20% threshold that instructors use to trigger manual review.
Winnowing's strength is speed and scalability; its weakness is that it operates on tokenized text in a linear window. Rename every token, and the windows look nothing alike. Reorder code blocks, and fingerprint sets diverge. MOSS is remarkably good at catching the lazy copy-paste, but it isn't designed to resist deliberate structural disguise.
MOSS reports similarity percentages, not a plagiarism decision. But if your threshold is 20%, Rung 3 transformations push the median below it. You're now missing cases a human reader would identify as derivative in 90 seconds.
Approach 2: Token-Based Matching with Suffix Trees — JPlag
JPlag takes a different approach: it tokenizes the source code — collapsing identifiers, literals, and operators into a token stream — and then constructs a generalized suffix tree over all submission pairs to find maximal identical token subsequences. Two submissions that share long token runs get a high similarity score. Because tokenization normalizes variable names, renaming has no effect: JPlag sees IDENTIFIER ASSIGN INTLITERAL PLUS IDENTIFIER regardless of what the student called the variables.
At Rung 1, JPlag's median similarity held at 97%. Renaming is a non-issue. At Rung 2 — method reordering — similarity dipped slightly to 91%: the suffix tree captures the long token runs within each method, and method boundaries aren't considered structural breaks. Rung 3 (loop swaps) caused a more noticeable drop to 74%, because the token stream for a for loop differs materially from a while loop — FOR LPAREN ... vs. WHILE LPAREN .... Still well above typical thresholds.
The real trouble started at Rung 4. Block extraction into a new method introduced a function call token in the original site and a new token stream for the extracted method body. JPlag's longest common subsequence shrinks, and similarity fell to a median of 52%. For pairs where the extracted block was substantial (15+ lines), similarity often landed in the high 30s — ambiguous territory where instructors balk at filing an integrity report.
JPlag is robust against renaming and moderate reordering, but it has no understanding of program structure beyond the linear token stream. Extract a block, and JPlag sees a discontinuity — even though the block's logic is unchanged.
Approach 3: AST Diffing and Structural Hashing
The third technique, and the one that forms the backbone of Codequiry's detect code plagiarism engine alongside its token layer, is abstract syntax tree (AST) comparison with structural hashing. Instead of treating code as text or a flat token sequence, the parser builds a tree where nodes represent syntactic constructs — class declarations, method bodies, if statements, loops, expressions. The detector then hashes subtrees and uses a tree-diffing algorithm to find the largest isomorphic subtrees shared between two submissions.
Running Codequiry's engine on the same pairs produced a starkly different profile. At Rung 1 and Rung 2, similarity held at 99% — AST structure is invariant to identifier names and declaration ordering. At Rung 3, loop-type swaps, similarity dropped only to 93%. The tool can be configured to treat for and equivalent while constructs as semantically similar when control-flow analysis is enabled. At Rung 4 — block extraction — the AST matcher identified the extracted method body as a structurally identical subtree, now located in a different method declaration node. Similarity remained at 88%.
At Rung 5, where statement reordering and declaration splitting occurred, the tree diff still found large common subtrees, though some fragmentation appeared. Median similarity held at 81%. In no case did a pair at any rung drop below the default 60% flagging threshold.
// Original snippet (before extraction)
public void assignSeat(int row, int col) {
if (seats[row][col] == null) {
seats[row][col] = passenger;
passenger.setSeat(row, col);
updateManifest();
logAssignment(passenger);
}
}
// After extraction (Rung 4)
public void assignSeat(int row, int col) {
if (seats[row][col] == null) {
seats[row][col] = passenger;
helperA1(passenger, row, col);
}
}
private void helperA1(Passenger p, int r, int c) {
p.setSeat(r, c);
updateManifest();
logAssignment(p);
}A linear token-based checker sees the introduction of a new method and a shortened call site as a break. The AST comparison sees the same statements, just relocated under a new MethodDeclaration node. The tree isomorphism is preserved.
Where Each Approach Breaks Down
No single technique is bulletproof. Winnowing fails at renaming and reordering. Token suffix trees fail at structural extraction and loop restructuring. AST diffing is robust against all of these, but its weakness is lexically-identical but syntactically-different rewrites — for example, a student who replaces a clean recursive solution with an iterative stack-based equivalent. Both produce correct output; the ASTs share almost no structure. Only a behavioral or semantic analysis — still an active research area — would catch that.
AST diffing is also computationally more expensive than winnowing or token-based comparison. Parsing thousands of submissions and building trees is O(n) per file, but tree-diffing is O(n²) naively. Codequiry's engine uses hash-based subtree fingerprinting to prune the comparison space, making it tractable for cohorts of several hundred students without overnight processing.
False positives are a concern with any technique. AST matching can over-flag submissions that both follow a rigid template or a prescribed class hierarchy, like a GUI assignment where the instructor provides a skeleton. This is where contextual thresholding matters — Codequiry's dashboard lets instructors exclude common framework code and adjust sensitivity per assignment, something neither MOSS nor JPlag expose natively.
Combining All Three: The Hybrid Stack in Codequiry
The real lesson from the ladder experiment isn't that one algorithm wins — it's that a stacked approach catches cases no single algorithm can. Codequiry's code plagiarism checker runs token fingerprinting, winnowing-style text comparison, and AST structural hashing in parallel, then merges the signals into a unified similarity report. When two of the three layers agree — say, AST similarity above 80% and token similarity above 50% — the confidence is extremely high. When only one layer fires, the report flags it for human review with a clear explanation.
Consider a real pair from the experiment at Rung 5: the winnowing layer registered 11% similarity (below any reasonable threshold), the token layer came back at 47% (ambiguous), but the AST layer produced an 83% match with the extracted-method transformation clearly highlighted. A single-technique checker would have let this case through. A stacked checker gave the instructor everything needed to make a ten-minute determination.
And for instructors who now face the additional challenge of AI code detector signals layered on top of peer-to-peer copying — students who generate a solution with ChatGPT, manually refactor it, and submit — the stacking principle extends further. Codequiry's AI-detection layer (trained on LLM output signatures like uniform complexity, low "burstiness" in token entropy, and stylistic smoothness) can be run on the same submission, giving a multi-axis view: Is this code similar to a peer's? Is it similar to online sources? Was it likely machine-generated? The answers often intersect in revealing ways.
Practical Takeaways for Instructors and Engineering Leads
If you're running a CS course or managing a team where code originality matters, the refactoring-ladder results suggest a few concrete actions:
- Don't rely on MOSS alone if you have students who know what MOSS is. A half-hour of renaming and method extraction can drop similarity below 15%.
- JPlag is a solid upgrade for renaming resistance, but expect false negatives when students extract functions or restructure loops. Use it with a lower threshold — 40% — and plan for manual review.
- AST-based comparison is the strongest single technique for detecting disguised plagiarism, but you need a tool that appropriately handles template code and lets you tune sensitivity.
- Stack signals. When possible, use a platform that combines multiple algorithms and cross-references with web sources and AI-generation indicators. The incremental cost is small; the risk of a missed case at an honor hearing is not.
The arms race between students who disguise code and the tools that find it isn't new, but the sophistication of both sides is accelerating. Knowing the algorithms — and their blind spots — keeps you one rung ahead.
Frequently Asked Questions
Can students beat MOSS by renaming variables and adding comments?
Yes, significantly. Winnowing fingerprints are sensitive to identifier names. Global renaming can drop MOSS similarity from near 100% to under 40%. Comments are typically stripped, so adding them has no effect.
Does JPlag catch code that has been split into smaller methods?
Not reliably. JPlag's token-based suffix-tree approach sees the extracted method as a separate token stream, breaking long identical subsequences. Scores often drop to 40–55% depending on the extraction size.
Is AST-based detection foolproof against refactoring?
No. AST matching handles renaming, reordering, and extraction well, but it struggles when the structure changes meaningfully — like replacing recursion with iteration. Semantic-equivalence detection remains an open problem.
How does Codequiry handle template code from instructors?
Codequiry's dashboard allows instructors to upload template or starter code files. The engine subtracts those AST subtrees and token sequences from similarity calculations, so that shared framework code doesn't inflate match scores.
Ready to see how your own submissions fare across multiple detection layers? Run a free scan with the code plagiarism checker used by universities worldwide.