Last spring, I pulled the similarity report for a 412-student data structures course and found 27 submissions above a 60% peer-match threshold. Eight were near-identical copies. Three were students who had worked together on the same whiteboard and submitted separate files. One had copied from a previous term's repository. The rest were false positives caused by the autograder boilerplate. The scores alone told me only the first part of that story. Interpreting code similarity scores in a large course means separating the signal from the scaffold.
A similarity score is a measure of structural overlap between one submission and another source. It is not proof of academic misconduct. It becomes useful only when you know what the tool counted, what your assignment's baseline looks like, and which scores deserve a human's time.
What a Similarity Score Actually Counts
Most code similarity tools do not compare text the way a string diff does. They normalize comments, whitespace, and identifier names, then compare some abstraction of the remaining structure. MOSS, the Stanford tool many CS departments still rely on, hashes overlapping token n-grams and uses winnowing to select a sparse fingerprint set. JPlag constructs a token string and uses greedy string tiling to find maximal matching substrings. Codequiry combines token-based matching with AST comparison and web fingerprinting, which catches refactors that a raw token stream will miss. A rename-and-reformat pass that clears MOSS can still leave an AST shape that is nearly identical.
For a concrete example, if one student renames all variables in a Java linked-list implementation and another reorders two methods, the token stream changes enough to lower a pure token score. The AST comparison still sees the same class structure, the same method bodies, and the same control flow. That detail matters because many students assume changing variable names is enough. It is not.
A similarity score is a ranking signal, not a verdict. The only thing worse than missing a real case is sending a student to conduct review on the basis of shared boilerplate.
When I first started reviewing reports, I ran MOSS on a CS2 lab and manually compared a code plagiarism checker side by side. The MOSS top pair matched at 88%; the Codequiry peer score was 91% but also surfaced a public GitHub source for the same lab. That web match changed the conversation with the student, because the original source was from the previous semester's instructor repository, not another enrolled student.
The Distribution You Should Expect in a Real Course
In fall 2023, I tracked peer-max scores for 412 Java 17 submissions on a 250-line maze solver. The median peer max was 24%, the 75th percentile was 53%, and the 90th percentile was 71%. Those numbers are not alarming. The floor is set by the assignment itself: every student had the same starter code, the same required method signatures, and the same autograder tests. A score under 15% is usually noise.
| Peer max bucket | Submissions | Share | Typical review outcome |
|---|---|---|---|
| 0 to 14% | 183 | 44.4% | No review, mostly starter code |
| 15 to 39% | 97 | 23.5% | Skim if web or AI score high |
| 40 to 69% | 62 | 15.0% | Review with side-by-side diff |
| 70 to 100% | 70 | 17.0% | Full review, check collaboration policy |
The table reflects one course, one language, and one assignment. I would not treat these ranges as universal. We have not tested the same thresholds at a different institution or in a course smaller than 80 students.
High scores do not automatically mean misconduct. In the 70 to 100% bucket, eight were near-identical copies, but eleven came from students who had attended the same lab section and had been explicitly allowed to discuss the algorithm on a whiteboard. The report could not see the whiteboard.
Calibrating Code Similarity Scores by Assignment Type
A threshold that works for a 40-line CS1 lab is wrong for a 900-line compiler project. Short assignments leave little room for structural divergence, so high scores are common even with independent work. Long projects produce more variation, and a 20% match on the full project can mean entire files were copied. The table below is the one I now start from, then adjust after the first scan.
| Assignment type | Typical size | Peer review trigger | Web review trigger |
|---|---|---|---|
| CS1 lab | 40 to 80 lines | 55% | 25% |
| Data structures project | 500 to 1000 lines | 25% | 15% |
| Compiler or systems project | 2000+ lines | 18% | 10% |
Every threshold assumes you have already excluded instructor-provided starter code. A 28-line linked list scaffold can produce a 70% match between two students who did not speak to each other all semester. I keep a small list of scaffold file paths in the check settings, and I update it every time a new assignment ships.
A source code plagiarism checker that returns peer, web, and per-file source matches in the same report makes those thresholds workable, because you can quickly see whether a match came from another student or from a public GitHub repository.
Triage Order for a 300-Submission Report
If you open a report and start at the top of a raw similarity list, you will spend the first hour on the wrong cases. I sort by peer max descending, then web max, then AI score. The highest peer score is not always the most serious; a 55% match on a 600-line project can hide copied core logic, while a 100% match on a 30-line warm-up is often two students sharing boilerplate.
def triage(peer_max: float, web_max: float, ai_score: float) -> str:
if peer_max < 0.15 and web_max < 0.10 and ai_score < 0.30:
return "none"
if peer_max >= 0.70 or ai_score >= 0.75:
return "first"
if peer_max >= 0.40 or web_max >= 0.25:
return "second"
return "later"
In the 412-student CS2 course, the `first` bucket usually held 17 to 24 submissions. That is a reviewable afternoon for two TAs if the report already groups the matched regions. The `second` bucket held another 30 to 40, most of which were resolved by opening the side-by-side view for 60 seconds. The `later` bucket was ignored unless a student had a pattern of late submissions or prior conduct history.
Codequiry's smart review queue ranks submissions by a cohort outlier score rather than raw percentage. That is more useful than sorting by highest raw score, because raw scores treat a 55% match in a 50-line lab and an 800-line project the same way.
When the AI Score Sits Next to the Similarity Score
AI-generated code changes the meaning of a similarity score. In spring 2024, two students in a Python 3.11 course submitted 64% peer similarity and both had AI scores of 72%. Neither student copied from the other. Both had prompted GitHub Copilot to generate a graph edge-list parser, and the model emitted nearly the same code. A peer-only checker would have framed that as collusion. The combined report from Codequiry showed the AI column, which changed the conversation entirely.
AI detection is not the same as similarity detection. A file can be 5% similar to every other student and still be 90% AI-generated. That is why I now run both signals in the same pass. Codequiry's AI code detector estimates the probability that a file was generated by a model rather than typed by a human, and the per-file view breaks down the indicators. It is not proof either, but it gives the right follow-up question: where did this function come from?
When both scores are high, I do not assume cheating. I ask the student to walk through the code in a short office-hours conversation. In several cases the student could explain the code and had used Copilot as an autocomplete tool without realizing how much of the file the model had filled in. That is a teaching moment, not a conduct case.
False Positives That Deserve a Second Look
The most common false positive in our courses is shared starter code. In fall 2023 I forgot to exclude starter code templates before running the first lab scan and burned two evenings reviewing 19 false positives from a 28-line linked list scaffold. The second most common is standard textbook implementations: a resize method for a hash table or a quicksort partition will look nearly identical across every student who read the same section of CLRS. The third is autograder test scaffolding, especially in Java where JUnit test methods and import blocks can dominate short files.
A high web score can also be a false positive if the student's code is in the same open-source repo you linked in the assignment. We now add known tutorial and starter repos to the assignment's allowed sources list before scanning, so those matches don't flood the review queue.
One rule has held across three terms: never send a student to a conduct meeting on the basis of a score alone. A TA or instructor opens the side-by-side diff first, checks for unique constants or comment phrasing, and then decides whether a conversation is warranted.
Escalation Policy for Repeatable Outcomes
Consistency across TAs matters more than the exact threshold numbers. I ask every TA to follow the same sequence, which I print on the first slide of the grading kickoff meeting:
- Exclude starter code and autograder files before any scan runs.
- Run peer, web, and AI detection in one pass.
- Sort submissions into first, second, and later buckets using the triage function above.
- Open the side-by-side diff for every submission in the first bucket and for the top 10 submissions in the second bucket.
- Check for constant strings, unusual comments, or unique variable names that persist across the match.
- Document the decision in the course tracking sheet before sending anything to the instructor.
This sequence has moved our review time for a 400-student course from roughly three days to two afternoons. It also reduced the number of disputed cases, because TAs could point to the same evidence view the instructor would later see.
If you're rethinking your review queue this term, start with a plagiarism checker for code that separates peer, web, and AI signals. The thresholds and triage rules in this article only work if the score is transparent enough to tell those signals apart.