A teaching assistant at a mid-sized public university opened a pair of submissions last spring and found two Java programs that shared every control structure, every magic number, and every slightly odd helper function name, but almost no identifiers in common. The student on one side had written a for loop. The other had written a while loop. A plain text diff between the two files reported less than 20 percent overlap. The structural comparison reported 91 percent.
That gap is the whole story of modern code similarity analysis. Refactored code plagiarism detection works because the tools stopped comparing characters and started comparing meaning. Renaming a variable, reordering two functions, or converting a for into a while changes the text a great deal and the abstract syntax tree almost not at all.
What code plagiarism detection actually compares
Every detection engine sits somewhere on a spectrum from surface to structure. At the surface end you have raw text comparison and diff tools. One step up is tokenization, where the source is converted to a stream of lexical units and compared with k-gram or winnowing fingerprints. Further up is the abstract syntax tree, where the parser throws away formatting and reconstructs the program's grammatical shape. At the far end are graph representations like program dependence graphs and control flow graphs, which encode what the code computes rather than how it is written.
The choice matters enormously in a programming course, because students under pressure do not copy verbatim. They copy and then edit. Each edit moves the pair up or down this spectrum, and each engine has a ceiling on how far up it can still see.
The obfuscation ladder, from renamed variables to rewritten logic
The standard taxonomy here is old and still useful. Faidhi and Robinson published it in 1987, in a paper on detecting plagiarism in student Pascal programs, and it holds up well enough that most modern evaluations still reference it. They described six levels of modification, roughly ordered by effort:
- Changing comments and whitespace only.
- Changing identifiers: variable names, function names, class names.
- Reordering statements that are independent of each other, and reordering functions in a file.
- Changing data structures, for example swapping an array for a linked list or a
HashMapfor a parallel array. - Changing the structure of statements:
fortowhile,switchto a chain ofif, recursion to iteration. - Changing the underlying logic, so the program computes the same result by a genuinely different route.
Levels one through three are what students actually do. A 2019 replication study of the taxonomy across three institutions found that of submissions later confirmed as copied after an honor council review, more than 70 percent fell at level two or three. Level one is rare now, because nobody bothers to change only the comments. Levels five and six are where the argument gets interesting, and where an instructor's judgment has to enter the picture.
How winnowing and token fingerprints work
The best-known algorithm in this space is winnowing, published by Schleimer, Wilkerson, and Aiken in 2003 at SIGMOD and used by MOSS, the Stanford tool that has been the default in CS departments since the late 1990s. The idea is to break a token stream into overlapping k-grams, hash each one, and then select a subset of those hashes using a sliding window. The selected hashes become the document's fingerprint. Two documents that share a long run of tokens will share fingerprints, and the algorithm guarantees it will find any match of at least a certain length, which is a property most heuristics cannot claim.
Winnowing is fast, language-agnostic once you have a good lexer, and cheap to run over thousands of submissions. It is also blind to structure. Consider this pair:
public static int computeSum(int[] values) {
int total = 0;
for (int i = 0; i < values.length; i++) {
total += values[i];
}
return total;
}public static int addUp(int[] nums) {
int acc = 0;
int idx = 0;
while (idx < nums.length) {
acc = acc + nums[idx];
idx++;
}
return acc;
}Token-window methods catch the first kind of edit easily. Rename total to acc, drop the compound assignment, and the token stream still lines up over long stretches. Move to a while loop, though, and you have inserted and removed enough tokens to break the windows. A pure winnowing score on this pair typically lands in the 40 to 60 percent range, which is below most departments' action threshold.
How AST comparison catches refactored code plagiarism
Parse both programs into syntax trees and the picture changes. The compiler front end reduces both snippets to structurally equivalent nodes: a variable declaration, an accumulator initialization, a loop with a condition and an incrementing counter, an indexed access, an addition, a return. The while loop and the for loop are different nodes in the grammar, but they are close enough in the tree that a tree edit distance algorithm finds a small distance and a high similarity score.
Clone detection research has used this approach for a long time. Baxter and colleagues described an AST-based clone detector in 1998, and the technique has been refined continuously since. Roy and Cordy's 2008 survey of clone detection techniques catalogued more than thirty tools across text, token, tree, and graph families, and the general finding was consistent: as you move from text toward graph representations, recall on structurally modified copies goes up and runtime goes up with it.
The tradeoff is that tree comparison is sensitive to parser quality. A tool that parses Java with a well-maintained ANTLR grammar will be reliable; one that falls back to regex for an unfamiliar language will produce noisy trees and noisy results. This is why language coverage is a practical purchasing question rather than a marketing footnote. JPlag, developed at KIT in Karlsruhe, supported a handful of languages through its 2.x line and expanded past twenty after the 4.x release in 2022, and the difference in output quality between the two eras is not subtle.
Detection rates by obfuscation level
The table below synthesizes reported results from Bellon and colleagues' 2007 comparison of six clone detectors in IEEE Transactions on Software Engineering, the Roy and Cordy survey, and published evaluations of the 2022 Dolos tool from Ghent University. Figures are approximate and vary with corpus, language, and threshold settings.
| Obfuscation level | Text / diff | Token + winnowing | AST comparison | Graph / PDG |
|---|---|---|---|---|
| L1 Comments and whitespace | High | ~99% | ~99% | ~99% |
| L2 Identifiers renamed | Low | ~95% | ~97% | ~97% |
| L3 Statement reordering | Very low | ~65% | ~88% | ~90% |
| L4 Data structure swaps | Very low | ~45% | ~72% | ~80% |
| L5 Control structure rewrites | Negligible | ~30% | ~60% | ~75% |
| L6 Logic rewritten | Negligible | Low | Low | ~40% |
Two things stand out. First, the cliff sits between level two and level three for token methods, which is exactly where real student copying sits. Second, no single engine dominates the whole ladder. Graph-based comparison wins on deep restructuring and costs the most to run, often by an order of magnitude in wall-clock time on a large cohort.
The practical lesson from two decades of clone detection benchmarks is not that one representation wins. It is that a detector built on a single representation has a predictable blind spot, and students find blind spots faster than tool vendors close them.
Where every method breaks down
False positives are the part of this work that gets discussed least and costs the most. Structural comparison is very good at finding real copying and moderately good at finding innocent similarity, and the difference is often context the algorithm cannot see.
- Starter code and lab skeletons. If the instructor hands out a 60-line scaffold, every submission matches every other submission on those 60 lines. Good tools let you exclude a base file or mask matched regions contributed by the assignment template.
- Boilerplate that everybody writes. In Java,
public static void main(String[] args)and aScannersetup appear in thousands of files an hour. Setting thresholds without accounting for common-line frequency will flag entire cohorts. - Framework and IDE templates. Generated constructors, getters, and test scaffolds inflate similarity between submissions that share nothing else.
- Genuinely convergent solutions. A one-line fizzbuzz has one correct answer. Short programs produce high baseline similarity and should be graded with that in mind.
The fix is not a better algorithm. It is frequency filtering, assignment-level configuration, and a human in the loop. When instructors skip that step, the tool's output becomes noise and the department stops trusting it. One department found this out the hard way when its rollout slipped a full semester because the submission portal exported filenames containing spaces and the ingest step silently dropped fourteen files, producing a cohort report with a subtly wrong denominator that took two weeks to diagnose.
Why LLM-generated code needs a different signal
Structural comparison answers one question: did these two submissions come from a common source? It does not answer the question more instructors are asking now, which is whether a single submission came from a large language model at all. Those are different problems and they need different signals.
Copied code has a sibling. Generated code often does not. A student who pastes a ChatGPT response into an assignment produces a file with no near neighbor in the cohort, an unusually regular style, and a tendency toward correct but slightly over-general solutions. A Python function from a model might look like this:
def find_duplicates(items: list) -> list:
"""Return elements that appear more than once, preserving order."""
seen = set()
duplicates = []
for item in items:
if item in seen and item not in duplicates:
duplicates.append(item)
seen.add(item)
return duplicates
That is clean, idiomatic, handles the edge case, and includes a docstring. None of those facts proves anything. Plenty of second-year students write exactly that. The signal is statistical and comparative: how the file's token distribution sits against the rest of the cohort, how consistent the comment density is, whether the student's earlier submissions look like the same person wrote them. Deployed alone, any one of those measures is weak. Combined with peer similarity and web-source matching, they become a coherent picture, which is why stacking engines matters more than picking the single best one.
Choosing and combining engines
MOSS remains free and effective, and it is still the default in many departments. It is also a web form that returns an HTML page of match links, it is rate-limited, and it has no concept of a course roster, a web-source check, or an AI signal. JPlag has a cleaner architecture and an open-source codebase, and Dolos pushed the field toward language-agnostic parsing with tree-sitter, which is a genuine advance. All three assume the operator has somewhere else to keep submissions, results, and student records.
That is the gap a commercial code plagiarism checker for teachers is built to close. Codequiry runs token, AST, and fingerprint comparison together rather than offering one representation and calling it a day, which matters specifically at obfuscation level three and above. It checks submissions against peers, against the open web and GitHub, and against a model trained to score AI generation, and it presents all three on one risk score rather than three disconnected reports. The web and GitHub matching is worth calling out for courses that assign well-known problems, because copied-adjacent code from a public repository will not appear in any peer comparison.
For engineering teams rather than classrooms, the same engine is reachable through a REST API and CLI, so a contractor's delivered module or a candidate's take-home can be run through the same checks inside an existing pipeline. A comparison of how this stacks up against the academic default is worth reading if MOSS is your current baseline: Codequiry vs MOSS covers the differences in coverage, reporting, and workflow in detail.
One honest limitation applies to every tool in this category. None of them decide anything. A 91 percent structural match is evidence, not a verdict, and the two most common causes of a high score that are not misconduct are shared starter code and a shared tutor. Reviewing the diff yourself remains the final step, which is why the review interface matters as much as the score. If you want to see what that workflow looks like on your own submissions, you can run a source code plagiarism check on a sample cohort before committing to anything.
Frequently Asked Questions
Does renaming variables defeat code plagiarism detection?
No. Identifier renaming is level two on the Faidhi-Robinson ladder and both token-based and AST-based detectors handle it reliably. Winnowing fingerprints are built from token patterns that ignore identifier spelling, and tree comparison operates on grammar nodes rather than names.
What is the difference between token-based and AST-based similarity?
Token-based methods compare sequences of lexical units using k-gram hashing and winnowing, and they are fast and portable. AST methods parse the program and compare tree structure, which survives reordering and control-structure rewrites that reliably break token windows. Most strong detectors run both.
How accurate is refactored code plagiarism detection?
Published benchmarks put structural detectors in the 85 to 95 percent range for renamed and reordered code, dropping sharply for fully rewritten logic. Accuracy also depends heavily on configuration: excluding starter code and filtering high-frequency boilerplate does more for precision than swapping algorithms.
Can a plagiarism checker tell whether code was written by ChatGPT?
Not from similarity alone. AI generation is a separate signal, typically derived from token distribution, style consistency, and comparison against a model of human-written student code. The strongest results come from combining AI scoring with peer and web-source checks in one report.
Structural comparison has quietly become the load-bearing wall of academic integrity in computer science, and it works well enough that the interesting failures now happen in configuration and review rather than in the algorithms. Start a check on your next assignment and see which level of the ladder your current tooling actually reaches.