The clump showed up about ten days after the deadline. Twenty-three of 412 submissions in a second-year data structures course shared a Dijkstra implementation that differed by fewer than three tokens. Not three percent. Three tokens. Pairwise similarity ran 94% to 100%, and the review queue surfaced them together because their cohort outlier scores were nearly identical.
The obvious reading is a group chat with 22 participants. That reading was wrong, and untangling it took three weeks of my time. Distinguishing AI-generated code from peer plagiarism turned out to be a measurement problem before it was a policy problem.
What does a 23-way code match actually look like?
The assignment was conventional: read a weighted graph from a file, run Dijkstra, print shortest paths to six fixed destinations. Roughly 120 lines of Python, a 900-line spec, two weeks. Four lab sections, two campuses.
We checked the social graph first, because a 23-way peer match with no transmission path is a contradiction. Eleven of the 23 had never shared a section. Two were evening-division students who worked full time. There was no shared Discord for the course that anyone would admit to, and the submission timestamps were spread over 61 hours in no meaningful order. Whatever produced that similarity, it did not travel from student to student.
The same semester gave us a useful control. Five submissions in the same cohort came back at 88% to 97% similarity, and that one was copying. A single origin, four downstream copies, and a clear directional structure once you looked. Same tool, same parameters, two completely different shapes.
We ran MOSS alongside our own pipeline with default parameters (minimum match of 20 lines). MOSS found the 23-way clump too, which should surprise no one, and also surfaced the five-person copy chain. Both engines agreed on which submissions were similar. Neither one told us why, and that gap is the whole story. If you want the longer comparison of how these engines differ on the same corpus, it's worth reading the Codequiry vs MOSS breakdown before you pick a default parameter set.
Why convergent AI output looks like peer plagiarism
Token-based similarity works by fingerprinting. Schleimer, Wilkerson, and Aiken described the winnowing scheme in 2003: hash every k-gram (MOSS uses k=50 for text, and code gets normalized first), select the minimum hash in each window of w, and store the survivors in a bucket keyed by hash. Similarity is then the size of the intersection between two documents' fingerprint sets.
Read that algorithm closely and you'll notice it has no concept of direction. It measures overlap, not descent. Direction has to be inferred, and the inference comes from containment asymmetry: if A copied B and then added two helper functions, B's fingerprints are nearly a subset of A's, with a few extras. Jaccard similarity hides this because it normalizes both sides.
Across the 23-way clump, containment ratios were symmetric within 1 to 2 percentage points in every direction. There was no origin, no leaf, no tree. That is a clique, and cliques are almost never how twenty-three undergraduates coordinate plagiarism.
The AST layer said the same thing. Baxter and colleagues described subtree hashing back in 1998, and both JPlag 4.0 (2023) and Dolos (2022) build on tree comparison in their own ways. All three approaches, run over the same set, converged: the 23 files had near-identical abstract syntax trees with statement ordering preserved. Reordering statements, renaming every variable, and rewriting the comments would not have changed that, because the AST does not care what you called your variables.
An AST match tells you two programs are structurally the same. It does not tell you whether one program is the ancestor of the other. Those are different questions and they need different evidence.
How to tell AI-generated code from peer plagiarism in the same cohort
The evidence that actually settled it lived in the identifiers. I pulled the naming distribution for the whole cohort and compared it to the 23.
The baseline cohort, meaning the 389 submissions outside the clump, used short, inconsistent local names: n, cnt, cur, pq, dist, d in the same file, sometimes the same variable spelled two different ways. Mean local identifier length was 4.1 characters. Nineteen of the 389 had at least one single-letter identifier used as a loop bound.
The 23 used current_distance, priority_queue, neighbor, distances. Mean identifier length 11.3 characters. Zero single-letter locals in any of them. And the comments were complete sentences ending in periods, restating the line directly beneath them.
def dijkstra(graph, start):
distances = {node: float('inf') for node in graph}
distances[start] = 0
priority_queue = [(0, start)]
while priority_queue:
current_distance, current_node = heapq.heappop(priority_queue)
if current_distance > distances[current_node]:
continue
for neighbor, weight in graph[current_node].items():
distance = current_distance + weight
if distance < distances[neighbor]:
distances[neighbor] = distance
heapq.heappush(priority_queue, (distance, neighbor))
return distancesThat is the canonical answer to "implement Dijkstra in Python" from essentially every general-purpose model shipped in 2024 and 2025. Twenty-three students asked a similar question and got a similar answer, because the specification left little room and the models are trained on the same GitHub corpus. There was no copying. There was convergence.
Two smaller tells. Six of the 23 imported from typing import Optional and never used it, a harmless artifact of generation. And two had the same typo in a docstring, which briefly looked damning until we noticed the typo appears in a widely mirrored tutorial that both models had likely absorbed.
Perplexity scores do not transfer from prose to source code
This is where most AI detectors get into trouble on code, so we stopped using the text machinery for it.
GLTR (Gehrmann, Strobelt, and Rush, 2019) works because natural language has meaningful next-token entropy. Human prose wanders; model prose doesn't. Hindle and colleagues made the uncomfortable observation back in 2012 that source code is radically more repetitive than English, which is exactly why statistical language models work so well on it. That same repetitiveness flattens the perplexity gap. A for loop is low-perplexity no matter who typed it.
We measured it anyway. On the 23 files, a perplexity-based score ranged from 0.71 to 0.94 on raw input and dropped into the 0.55 to 0.68 band once we formatted and stripped comments. A 40-point swing from whitespace normalization is not a signal you can put in front of an honor council. I would not rely on perplexity on code as anything but a weak prior, and we've stopped reporting it as a standalone number.
The useful surface is the natural-language residue inside the file: comments, docstrings, identifier naming, and how consistently the author applies their own conventions. Those carry actual entropy. The structural regularity of the code carries a different, complementary signal, and it's one of the things the AI code detector in Codequiry weights when it scores a submission, alongside the token and AST work.
The three-pass workflow that separated the two cases
We ran three checks and kept the outputs separate on purpose. Merging them into a single "integrity score" is the mistake I see most often, because it makes a convergent AI clump look like a cheating ring and destroys the evidence you need to tell them apart.
| Signal | Threshold | Flagged | What it actually meant |
|---|---|---|---|
| Peer similarity, token + AST | ≥ 85% | 38 | 23 AI clump, 5 copy chain, 10 unrelated pairs |
| Web and repository match | ≥ 60% line coverage | 7 | 2 GitHub repos, 3 Stack Overflow answers, 2 prior-year course solutions |
| AI generation score | ≥ 0.80 | 71 | Broad, includes AI-assisted work that was then substantially rewritten |
| All three at once | overlap | 4 | Genuinely worth a conversation |
Thirty-eight peer matches sounds alarming. Ten of those were students who had done exactly what we asked, namely discussed the problem in office hours and then written very similar driver code, which any honest comparison engine will flag and any honest instructor will dismiss in under a minute. The point of a peer similarity report isn't the number of flags. It's whether the report gives you enough structure to triage them fast.
We ran the whole pipeline through Codequiry that semester, mostly because the three scores arrive as separate columns with the match sources attached, rather than collapsed into one bar. That mattered more than I expected. A source code plagiarism checker that returns one number forces you to guess at mechanism, and mechanism was the entire question. The web pass earned its keep differently: three of the seven web matches traced to Stack Overflow answers that predated the assignment by years, which is a citation problem and not an integrity problem, and the report labeled the source so we could say so.
What we did about the 23 students
The copy chain was straightforward. Five students, honor code process, standard sanctions, and the containment asymmetry in the report made the conversation short. Nobody argued.
The 23 got split into two groups once we had the evidence assembled. Four of them had clearly also copied from each other on top of the AI output: submission timestamps within 90 seconds, identical deviations from the canonical snippet, and one shared misspelled filename. Those went to the honor council as a small case. The remaining 19 faced a policy issue, not a cheating issue. The syllabus said work must be your own and did not mention AI at all, which is a syllabus failure before it's a student failure.
We gave those 19 a redo with an oral walkthrough of their own code. All 19 could explain the algorithm at a reasonable level. Six could not explain their own edge-case handling, which is a pretty reliable sign they had never debugged it, and those six went to the honor council with the AI report attached as context rather than as the charge. Nine of the 19 said some version of "the code was right, so I didn't think about it," which is worth sitting with.
Where this method breaks down
The honest limits. This is one assignment, one cohort, 412 submissions, and one discipline (student Python). The identifier-entropy threshold we leaned on, mean local name length above 8 characters with zero one-letter locals, will flag the occasional tidy student who names things well and writes full-sentence comments because that's how they were taught. It's a heuristic and we used it as one, never as a solo basis for a finding.
The hardest case remains AI-assisted work. A student who asks a model for a starting point and then genuinely rewrites it looks nothing like the 23-way clump and nothing like a copier, and no tool I've used separates "used AI as a reference" from "generated and lightly edited" with real reliability. We've run this pipeline across roughly 6,000 submissions now and I would not push those thresholds any further without retuning per course. A first-semester C course and a senior compilers elective will not share thresholds, and pretending otherwise is how you generate false positives that burn your credibility with students.
The clump itself was the easy part. Twenty-three identical Dijkstra implementations is a loud signal pointing at a real problem, just not the problem it appears to be. If you want to run the three passes on your own submission set and see the separation for yourself, you can run a peer, web, and AI check on the same batch and read the scores as three columns instead of one verdict.