The first time I plotted every similarity score from a single class, I expected a carpet near zero and a few spikes at ninety-something. I got a fat hump centered at 41 percent and no clean valley anywhere. That plot is why I stopped treating a code similarity threshold as one number you set once in August and never revisit.
Four semesters and 12,400 submissions later, here's what I actually do: I set the threshold per assignment, I read the score next to an AI signal, and I budget roughly four minutes of human judgment for every pair I pull into review.
What a similarity score is actually counting
Every tool in this space normalizes your students' source before it compares anything. Comments come out. Whitespace comes out. Identifiers get canonicalized, so total in one submission and sum in another become the same symbol. What's left is structure, and the tools measure structure differently.
MOSS, which Mike Schleimer, Alex Aiken, and colleagues at Stanford published in 2003, hashes overlapping k-grams and keeps a winnowed subset of fingerprints. JPlag 4.x, the 2023 rewrite out of KIT, aligns token sequences with greedy string tiling and reports an average similarity across matched regions. Dolos, released in 2022 by the ETH Zurich group, added tree-based comparison on top of tokens. Codequiry runs token, AST, and fingerprint matching together, which matters mostly for the adversarial cases I'll get to.
The point for a grader is simpler than the mechanics: a score of 60 percent does not mean 60 percent of the work was copied. It means 60 percent of the comparable structure lined up after normalization. In a fifteen-line program, that's a rounding error away from coincidence.
The innocent floor differs by language
This is the part nobody tells a new TA. Two students who have never spoken, working independently and correctly, will still land at a surprisingly high similarity in some languages. I pulled the distribution from every pair within each cohort, discarded the top 5 percent, and took the median of what remained as my baseline.
| Assignment type | Language | Median innocent pair | 90th percentile |
|---|---|---|---|
| First functions and I/O | Python | 38% | 61% |
| Loops and arrays | Java | 34% | 58% |
| Pointers, linked lists | C++ | 22% | 44% |
| Binary search trees | Java | 47% | 72% |
| Recursion drills | Python | 29% | 52% |
Those numbers come from our own corpus, and I cross-checked two of the four semesters against MOSS before I trusted them. Your floor will move; the shape won't.
Java and C++ sit high because the language makes everyone type the same scaffolding before they write a single line of logic. Here is the honest content of a "sum n numbers" submission in C++:
#include <iostream>
using namespace std;
int main() {
int n;
cin >> n;
int total = 0;
for (int i = 0; i < n; i++) {
int x;
cin >> x;
total += x;
}
cout << total << endl;
return 0;
}Strip the comments and the whitespace and count what's left. Maybe a dozen tokens carry real decisions. The other thirty are the price of admission. Two correct submissions will clear 70 percent on a token matcher without either student having done anything wrong. Python hides this better because the boilerplate evaporates, but the same effect shows up in tiny functions where there is exactly one reasonable spelling. A binary search tree in Java has a canonical shape, and every cohort produces forty of them.
The threshold is not a property of the tool. It's a property of the assignment.
Where I actually put the line
I run two numbers per assignment: a review floor and an investigate line. Anything under the floor I don't open. Between the floor and the investigate line, I skim. Above it, I read.
For assignments under about 30 lines where students get a starter file, the review floor is 55 percent and the investigate line is 70. For a full-program take-home with no skeleton, the floor drops to 40 and investigate to 60. For a data structures assignment with a provided class stub, I exclude the stub from the comparison entirely (every serious tool supports this; in Codequiry you mark the starter files when you create the check, and in MOSS it's the -b base-file flag that half the TAs I've trained forget about) and then the floor drops hard, because the remaining logic is short and highly constrained.
Two things I got wrong early. First, I used one threshold across a whole course, which meant my Python recursion drills generated sixty flagged pairs a week and my C++ pointers assignment generated none. Second, I treated a 100 percent match as proof.
It isn't. Last spring I opened a 100 percent pair and found two students who had both submitted the provided skeleton with an empty main that printed Not implemented. Neither had written a line. A score is a pointer to a file, not a verdict.
The two failure modes a threshold cannot catch
The false positive is annoying. The false negative is corrosive, and this is the one that keeps me up at night, because it's the case where the tool tells you everything is fine.
Straight peer-similarity comparison is bad at a specific scenario: one student copies from a source that isn't in your cohort. A GitHub repository. A Stack Overflow answer from 2016. A tutorial blog. Their peer score comes back at 12 percent, no one flags anything, and yet the submission has a perfect line-for-line ancestor on the open web. That's why I never run peer comparison alone. The web and repository check catches what the cohort view structurally cannot, and it's the difference between catching one student and catching the whole chain of them.
The AI wrinkle in mutual similarity
Here's the pattern that changed my workflow in 2024. I found 31 pairs with mutual similarity above 85 percent where neither student could identify the other, and in several cases they had never been in the same discussion section. Two students asked ChatGPT the same question in the same week and got back near-identical code. They didn't collude. They shared a source.
Peer similarity cannot tell those apart on its own. Reading it next to an AI score can:
| Peer similarity | AI score (both students) | Most likely explanation |
|---|---|---|
| 88% | 12% and 9% | Real collusion, or a shared web source. Check the web results. |
| 88% | 91% and 87% | Shared model, not a shared student. Handle as an AI-integrity case. |
| 14% | 78% | Single submission, model-assisted. Peer similarity is blind to this. |
| 95% | 3% and 4% | Almost always the provided starter file. Exclude it and re-run. |
That second row is not a plagiarism case, and if you discipline it as one you will lose the hearing. It's a policy case about what the syllabus allowed. Keep them separate. The AI code detector gives you a per-file probability and, more usefully for a conversation with a student, a written list of the indicators it saw.
The Tuesday night triage
Here is the actual sequence, with timings, because I've had three new TAs ask and hand-waving doesn't help.
- Export the peer report as CSV and open it in a sheet. Add three columns: decision, AI score, notes.
- Sort by peer similarity descending. Filter AI score above 60 into a separate tab, since those cases route differently.
- Work the list top-down. Four minutes per pair for a first pass: open the side-by-side diff, read the top three matched blocks, decide collusion / shared source / starter file / clean.
- Only the collusion pile gets a second read with the AI and web reports open next to it.
A 210-student class generates roughly 40 pairs above my investigate line. At four minutes each that's about two and a half hours, and I do it in two sittings because my judgment on the third hour is visibly worse. The code plagiarism checker does the sorting and the diff view; the reading is still mine.
One practical note from the trenches: in the fall 2022 pilot, our CSV export silently dropped the per-file column for any submission zipped with a nested folder, so for two weeks I was triaging by hand off the HTML report. Export a sample and eyeball it before you trust the column headers. That's true of every tool, not just this one, and MOSS results expire after 14 days, so if the sheet is wrong you can't just go back and re-download three weeks later.
What I'd tell a new TA in one minute
Build the baseline before you judge anyone. Plot the whole cohort's pairwise scores for the assignment, find your innocent floor, and set your review line above it. Read the score next to an AI signal and next to the web check, and treat the three as evidence rather than a verdict. When you pull a student in, show them the matched block, not the number. I have never had a productive conversation that started with "you scored 82."
The tools have gotten much better at this over four years. The reading hasn't gotten shorter. If you're setting this up for the first time and want one place where peer similarity, web matches, and AI scores land in the same report, that's the case for Codequiry over stitching MOSS output into a spreadsheet by hand. If you're evaluating options, I wrote up the comparison against MOSS from a grading workflow perspective.
Frequently asked questions
What is a good similarity threshold for code plagiarism?
There isn't one universal number. Establish your cohort's innocent baseline for that specific assignment, then set your review floor above it. In practice that lands near 55 percent for small Java assignments and closer to 40 percent for open-ended take-homes.
Why do two students who never talked have a 90 percent match?
Three common reasons: they both submitted the provided starter code, they both found the same online source, or they both prompted an AI assistant with the same assignment text. Check the AI score and the web results before you assume collusion.
Does a low similarity score mean the code is original?
No. Peer similarity only compares submissions inside your cohort. Code copied from GitHub, a blog, or Stack Overflow can score near zero and still be copied wholesale, which is why the web check has to run every time.
If you want to see how the three signals look side by side on your own assignments, start with a sample AI detection run on one past cohort and compare it against the peer scores you already have.