Two Februaries ago I sat on a hearing for a second-year student in our data structures course. His project, 214 lines of Java, matched a roommate's submission at 94 percent. He had an explanation ready, and it was the one we hear most often: "I renamed everything."
He had. Variable names, method names, even the order of two private helpers. What he had not changed, and did not seem to understand, was the structure underneath. A code plagiarism report is not a statement about variable names. It is a statement about shape, and shape is much harder to disguise than students believe.
That case was straightforward. Many are not. Over two terms on our college's academic-integrity committee I have reviewed somewhere north of sixty reports, and I have watched colleagues draw the wrong conclusion from good evidence and the right conclusion from thin evidence. What follows is how I read these documents now, and what I wish someone had told me during my first grading marathon with one open in a browser tab.
What a code plagiarism report actually measures
The tools most departments use fall into a few families, and knowing which family produced your report tells you what the number means. MOSS, which Alex Aiken released at Berkeley in 1994 and which is still running as of this writing, uses a winnowing fingerprinting algorithm over token sequences. JPlag, built at the University of Karlsruhe beginning in 1996 and now at version 5.x, leans on token sequences and greedy string tiling. Dolos, out of KU Leuven in 2021, adds tree-based comparison for languages where an AST is available. Codequiry's engine combines token and AST comparison with fingerprinting, and checks each submission against its peers, the open web, GitHub, and a set of AI-generation signals in one pass.
All of those approaches share a property worth understanding: they compare structure, not text. Renaming a variable costs a student almost nothing in similarity score. Reordering independent statements costs a little. Genuinely rewriting the algorithm costs a great deal. Consider two submissions from an intro course:
// Submission A
for (int i = 0; i < scores.length; i++) {
if (scores[i] > threshold) {
total += scores[i];
}
}
// Submission B
int idx = 0;
while (idx < values.length) {
boolean keep = values[idx] > cutoff;
if (keep) {
sum = sum + values[idx];
}
idx++;
}A text diff finds almost nothing here. A token-based tool with normalization finds a great deal, because after you strip identifiers and collapse the for-loop and while-loop into comparable control structures, these are the same program. This is exactly the case that motivated the original MOSS work, and it is still the case that catches the most students.
Reading the score against the cohort, not in isolation
The single most common misreading I see from new TAs is treating an absolute number as evidence. A 40 percent similarity score sounds damning until you learn that the median pair in that assignment scores 38 percent, because the assignment is a 30-line exercise with a required method signature and only one sensible loop. In a cohort like that, 40 percent is noise.
The opposite happens in capstone courses where students pick their own projects. A 62 percent match between two submissions that share nothing but a Maven boilerplate and a Maven-generated wrapper class is also noise, and it will show up on every report you pull.
What matters is the distribution. A report worth acting on shows one pair at 94 percent when the next highest pair in the class is at 31 percent. That gap is the signal, not the absolute value. This is why I prefer tools that rank submissions by how far they sit from the cohort rather than presenting a flat sorted list of scores. Codequiry's review queue does this by surfacing outliers first, and I have found that it changes the reading experience: instead of scanning 180 rows looking for a big number, you start with the twelve that deserve attention and work downward.
One practical note. Pull the report before you read a single submission. If you grade first and run the check afterward, you will have already formed opinions about whose code you like, and that is a worse bias than any algorithm's.
Web matches, attribution, and the license question
Peer similarity gets the attention, but the web half of the report is where most of the genuinely interesting cases live. Students copy from Stack Overflow answers, from YouTube tutorial repos, from GitHub projects under MIT and GPL licenses, and increasingly from the first page of a Google search for the assignment prompt itself, which at some institutions returns last year's solution posted by a student who did not think anyone would look.
The report tells you where the code came from. It does not tell you whether that matters, and those are separate questions.
A web match is a provenance fact. Whether it is a violation depends on your course policy, and if your course policy does not say, you are about to have an argument in a hearing room.
If a student pulled a licensed snippet and left the attribution header intact, that is a citation, not plagiarism, and the conversation should be about scope of use, not cheating. If the header was stripped, you have two problems: an academic one and, for code destined for a public repository, a licensing one. I have seen a student project shipped to a public GitHub organization with a GPL-licensed parsing library inside it and no notice. Nobody in the department noticed until an alumnus did.
This is also the argument for checking against the live web rather than a fixed corpus. Any tool that compares only within a course misses the most common source of copied code in 2024 and 2025, which is not the person in the next seat. It is a repository indexed three years ago.
Where the AI score fits and where it does not
Our department added AI detection to the standard report in the spring of 2024. The rollout slipped three weeks because our LMS changed its export column order and the roster sync quietly dropped nine students, which is a story for another time and a reminder to verify your roster before your first check of the semester.
I want to be precise about what these scores establish, because I have seen them misused in both directions. An AI score is a statistical estimate of how likely a passage of code is to have been produced by a language model rather than typed by a human working through a problem. Detectors look at signals like token predictability, structural uniformity, comment style, and the presence of patterns that LLMs overproduce: defensive try/except blocks around operations that cannot throw, docstrings on three-line helpers, type hints on a script that will never be imported, and logging calls in a homework assignment.
def process_names(names: Optional[List[str]]) -> List[str]:
"""Process a list of names and return cleaned results."""
results: List[str] = []
try:
for name in names:
if name is not None:
results.append(name.strip())
except Exception as e:
logging.error(f"Error processing names: {e}")
return resultsA second-year student writing a fifteen-line homework function does not usually wrap it in exception handling, import Optional, and log failures. A model asked to "write a robust function" does exactly that. None of this is proof. A careful student who read a style guide can produce the same code, and I have had one do so.
The correct use of an AI score in a hearing is as a reason to ask a question, never as the answer. I treat a high score the way I treat a 70 percent peer match in a noisy cohort: it moves a submission to the top of my reading list. What I bring to the hearing is the follow-up, which is the student's ability to explain design decisions the model made on their behalf.
The practical difficulty is that AI-assisted work and fully AI-generated work sit on a continuum, and most of our students are somewhere in the middle. A student who asks a model to explain recursion and then writes their own function is doing something I would encourage. A student who pastes the prompt and submits the output is not. Detectors do not reliably separate those two cases, and any vendor who tells you otherwise is selling certainty they do not have. We have not tested our current thresholds past a few hundred submissions, and I would be cautious about importing our cutoffs wholesale into a different course.
The office hours conversation is the real test
Here is the part that does not appear in any tool's documentation. The most reliable plagiarism detector in a CS department is a fifteen-minute conversation about the student's own code.
Ask why they chose a HashMap over a TreeMap. Ask what happens if the input list is empty. Ask them to add one feature to their own function. A student who wrote the code will answer imperfectly but coherently, and they will usually tell you what they got stuck on. A student who did not write it will describe the code from the outside, as a reader rather than an author, and the tells are consistent: they talk about what the function "does" rather than what they were trying to make it do.
I have cleared students who scored 88 percent. In one case two students in a lab section had worked through the same textbook exercise together in the same room, which was permitted, and their solutions converged because the exercise had one natural solution. I have also flagged students whose report scored in the forties, because the conversation collapsed in the first three minutes. The report identifies candidates. The conversation decides.
What departments should write down before the semester starts
Most of the pain I have witnessed in hearings comes from policy gaps rather than detection failures. If your syllabus does not state whether AI assistance is permitted, you will be adjudicating a rule that did not exist when the assignment was posted. Write it down, put it in the assignment prompt, and be specific about the categories: no AI at all, AI for explanation only, AI permitted with disclosure, AI permitted without disclosure.
Second, publish the threshold. Students who know that submissions are compared against peers, against the web, and against AI-detection baselines behave differently in week two than students who find out in week twelve. I say this plainly on the first day of every course now, and my referral rate to the honor board has dropped by more than half since I started doing it. Some of that is deterrence and some of it is that students stop guessing where the line is.
Third, use a tool that produces a report you would be willing to hand to a student. That sounds like a low bar. It is not. I have received MOSS output as a URL to a plain-text table of scores with no per-line evidence, and I have had to reconstruct the case by hand before a hearing. A defensible report shows the matched regions side by side, cites the web sources with line numbers, separates peer similarity from web similarity from AI signals, and does not collapse three different kinds of evidence into one number. That separation is the difference between a document that supports a conversation and a document that starts a fight.
For departments still assembling a process, the code plagiarism checker for teachers workflow is a reasonable starting point, and if your institution is weighing a move away from MOSS, the comparison in Codequiry vs MOSS covers the operational differences honestly, including the ones that favor MOSS.
What I would tell a new committee member is this. The report is a map of where to look, and the map is drawn by an algorithm that has no idea what your assignment was for. A 94 percent match between two students who sat next to each other in lab and were told to collaborate is not a violation. A 94 percent match between two students who have never met is. The tool cannot tell those apart. You can, and that is the job.
If you want to see what a report with separated peer, web, and AI evidence looks like before your next semester starts, you can run a sample check on your own assignment files in about ten minutes.