The flags land on a Tuesday. Four hundred and twelve pairs, from 186 submissions on a single red-black tree assignment. Grades are due the following Monday at 5 p.m. The TA who owns this queue, a third-year PhD student I'll call Meera, has about fourteen hours of actual review time in that window, and roughly 40% of the flagged pairs are going to be noise.
The method she's built over five semesters is worth writing down, because it's mostly arithmetic and almost no intuition. She doesn't read code first. She sorts first. Here's how the queue gets built, filtered, and closed.
Start with the score distribution, not the top match
The instinct for a first-time TA is to open the highest similarity pair and work down from 100%. That's the wrong order. The top of the distribution is where the true positives are, but it's also where the judgment calls are hardest, and it burns the best hours of the week on the most contested cases.
Meera pulls the full pair list into a dataframe and plots it first. Across her department's 186 submissions, peer similarity scores on this assignment clustered with a median of 18% and a long tail: 31 submissions above 55%, of which 14 sat above 80%. Those 14 are the ones she reviews herself. The 55-80% band, 17 pairs, goes to two trained TAs with a written rubric. The remaining ~380 pairs get a 90-second automated look each.
The cutoff isn't magic. She set 55% after running a control: last fall she had the same assignment, and she submitted 20 known-independent solutions (written by prior-year students with a signed attestation) through the same scanner. The maximum pairwise score among genuinely independent solutions was 47%, and that was two students who'd both read the same CLRS chapter and both named their sentinel node NIL_LEAF. That control run is the whole justification for the threshold, and it's the first thing she shows when a student appeals.
The threshold is only defensible if you can show what innocent code scores. Two hours of control submissions buys you a semester of credibility.
Read the token overlap before you read the code
Surface text similarity is the weakest signal in the stack, which is why Meera's department doesn't run on it alone. Two students can score 70% on a naive line-matching tool because they both named variables left, right, parent and both followed the same lecture pseudocode. That's not plagiarism, that's a lecture.
What she looks at instead is the shape of the match after normalization. Renaming a variable, changing indentation, adding a comment block, or splitting one function into two should not move an AST-aware score more than a few points. If it does, the match is lexical and weak. If the score holds at 88% after a student has renamed everything and restructured two methods, that's a signal worth the next twenty minutes.
This is the practical difference between a text-diff comparison and a structural one. On the red-black tree assignment, one pair scored 61% on a line-based tool and 94% after AST and fingerprint comparison, because the student had renamed all identifiers, reordered two helper functions, and converted a while loop into a recursive helper. Same tree, same rotation logic, same variable semantics, different spelling. The source code plagiarism checker that catches that pair is the one doing token and tree comparison, not substring matching.
A short illustration of what "same tree, different spelling" looks like in practice. Two submissions, one renamed and restructured:
# Submission A
def rotate_left(self, x):
y = x.right
x.right = y.left
if y.left is not None:
y.left.parent = x
y.parent = x.parent
if x.parent is None:
self.root = y
elif x is x.parent.left:
x.parent.left = y
else:
x.parent.right = y
y.left = x
x.parent = y# Submission B, renamed and split
def _promote_right_child(self, node):
child = node.right_node
node.right_node = child.left_node
if child.left_node is not None:
child.left_node.parent_ref = node
self._reparent(child, node)
child.left_node = node
node.parent_ref = childA text matcher sees almost nothing here. A structural matcher sees the same five-step rotation with the same guard clause in the same position. Meera treats the second score as evidence and the first as a hint.
Layer the web check separately from the peer check
Peer similarity and web similarity answer different questions and should never be summed into one number before review. Peer score asks "did this student work with another student in this cohort." Web score asks "did this student copy from GitHub, a tutorial, or a forum." The appropriate response to each is completely different, including in a disciplinary sense.
On the algorithms assignment, 22 submissions had a web score above 40%. Nineteen of those traced to one of three sources: a widely-copied GeeksforGeeks red-black tree article, a 2019 GitHub repository for a data structures course, and a Stack Overflow answer about fixup cases. Only three of the 22 also scored high on peer similarity. If she'd collapsed both scores into a single risk number, she'd have missed the distinction between "student copied from the internet" and "two students copied from the same internet source," which is a real and common pattern. Two students who both copied the same blog post will show up as a peer match too, and the right finding there is web plagiarism for both, not collusion.
That distinction matters for how the conversation goes. A web match gets a conversation about attribution and learning outcomes. A peer match at 94% with no web source gets a different conversation entirely. Same score, different file.
Where AI detection fits in the queue, and where it doesn't
Meera runs AI detection as a third, independent column, and she is deliberate about not treating it as evidence on its own. An AI score of 85% on a 200-line submission is a reason to look at that submission with more care. It is not a reason to open a case. The false positive rate on short, boilerplate-heavy files is high enough that she won't act on an AI score alone, and she says so in writing to her department.
What makes the AI column useful is correlation. A submission with high AI score and high peer similarity is usually two people using the same prompt, or one person sharing output. A submission with high AI score and a clean peer score but a distinctive stylistic register that doesn't match the student's earlier work is worth a fifteen-minute conversation, not an accusation. A submission with a high AI score and a matching web source to a "generated with ChatGPT" gist is the easiest case in the whole queue.
The AI code detector Meera's department uses produces a per-file score rather than a single submission score, which turns out to matter. On a five-file assignment, she routinely sees one file at 91% and four files at 12-20%. That pattern is usually a student who used a model for one hard function and wrote the rest, and the file-level view is what makes that visible. A single aggregate score would have hidden it.
Normalize against the cohort before you rank
The single biggest source of wasted TA hours is raw score ranking across a heterogeneous cohort. A 78% match between two students who both submitted 900 lines of near-identical scaffolding from a provided starter repo is not a finding. A 34% match between two students who wrote 200 lines each, from scratch, with unusual shared structure in the three hardest functions, absolutely is.
Meera's normalization is crude but effective: she subtracts the starter-code baseline from every pair before ranking. The department's provided skeleton accounts for a median of 22 points of similarity on this assignment, because everyone has the same class declaration, the same imports, the same four empty method stubs. Once that baseline is removed, the effective distribution compresses and the tail gets much easier to read. The 78% scaffolding pair drops to 56% effective. The 34% hand-written pair rises to the top ten.
| Band | Pairs | Effective score after baseline | Review owner |
|---|---|---|---|
| Critical | 14 | > 80% | Lead TA, full read |
| Elevated | 17 | 55-80% | Trained TA, rubric |
| Scaffolding | 63 | Baseline only | Auto-closed |
| Noise | 318 | < 45% | 90-second scan |
The automated closure of the 63 scaffolding pairs is the reason the week works at all. Without it, those 63 would eat a full day of review and produce zero findings.
Write the disposition before you write the email
Every pair that survives to a real review gets a one-line disposition written before any communication goes out. The vocabulary is fixed, and it's been the same for three semesters: no finding, shared source, collusion, insufficient evidence, refer. The label determines who acts next and what the student is told.
"Insufficient evidence" is the one that saves the most time and the one most TAs skip. If a pair sits at 62% and Meera can't articulate in a sentence why the overlap isn't explained by shared lecture material, it gets that label and closes. She keeps a running count: last spring, 41 of 380 reviewed pairs closed as insufficient evidence. That's roughly 11%, and it's a number worth reporting upward, because it's a direct measure of how much noise the tool is generating for a given assignment.
The written dispositions also survive appeals. Two grade appeals in the past three semesters, both resolved in the department's favor, both because the file had a dated disposition, the control-run baseline, and a side-by-side screen capture attached before the student was ever contacted. The code plagiarism checker for teachers workflow that makes this cheap is the part that matters operationally: the evidence needs to be captured at review time, not reconstructed three weeks later when a student's advisor asks.
What the numbers look like at the end
Across the 412 flagged pairs from this assignment: 14 critical, of which 9 confirmed as collusion and 2 as shared web source, 3 closed as insufficient evidence. The 17 elevated pairs produced 4 more findings. Total confirmed cases, 15 out of 186 submissions, or about 8%. That rate has been stable at 6-11% across six semesters on this course, which is the kind of stability that tells you the threshold is roughly calibrated rather than the kind that tells you the tool is perfect.
Fourteen hours, 15 findings, zero successful appeals. The important number isn't the 15. It's the 63 scaffolding pairs that got closed without a human reading them, and the 41 that got closed as insufficient evidence rather than escalated. A queue you can't empty isn't a detection program. It's a backlog with a dashboard, and by week three every TA stops opening it.
Meera's setup runs on a single tool that does peer, web, and AI checks against the same submission set, with per-file scores and a review queue that ranks by cohort outlier. That matters less for accuracy than for the review hours it saves: one export, one queue, one place to attach dispositions. If you're evaluating that setup, Codequiry's code plagiarism checker is the one her department standardized on, and the review workspace is where the triage actually happens.