The department meeting had run twenty minutes over when the chair put the numbers on the screen. Fall 2024, Introduction to Computer Science I, 412 enrolled students, five lab sections, one head TA who coordinated seven graders. MOSS had flagged 23% of the first programming assignment for peer similarity above the threshold the department had used for years. The chair paused. "MOSS gave us a floor," she said. "I think we need to look at the ceiling."
What followed over the next two semesters was a deliberate move from single-tool similarity checking to a layered detection workflow. The department combined peer code similarity, web source matching, and AI-generated code detection into one review process. The change did not produce a scandal. It did produce a clearer picture of what students were actually submitting, and it forced a hard conversation about where the line falls between legitimate assistance and academic misconduct.
Why a Single Similarity Tool Stopped Being Enough
Before describing the new workflow, I should define what we mean by layered detection. Most code plagiarism checkers answer one question: how similar is this submission to another submission in the same course? That is peer similarity. Tools like MOSS, JPlag, and Dolos do this well. They compare token streams, build fingerprints, and rank pairs of submissions by overlap. The problem is that they answer only that one question.
For years, that was sufficient in many CS departments, including this one. The first programming assignment in Intro to Computer Science I at this university asks students to compute a semester grade from weighted exam and homework scores. The expected solution is about forty lines of Python. When two students submit nearly identical code, MOSS catches it. When a student copies from a GitHub repository that is not in the peer set, MOSS does not. When a student asks an LLM to generate the solution and then changes variable names, MOSS often does not flag it either, because the code is not similar to any other student submission.
The department had suspected this gap. The chair mentioned that in the previous year, a student had been reported by a grader for unusual style rather than similarity. The student's code used list comprehensions in ways the course had not yet covered, and the comments read like documentation from a library. MOSS showed nothing above threshold. The case was resolved manually, but it took three hours of instructor time to trace the code to a public Stack Overflow answer. The department wanted a workflow that would surface those cases before a grader had to notice them by feel.
What Combined Peer, Web, and AI Detection Looks Like in Practice
The department adopted Codequiry after evaluating three options. They needed a tool that could check peer submissions, search the open web, and run an AI-generation probability model on the same upload. Codequiry's code plagiarism checker does all three in one check, which mattered more than the department expected, because a single submission often triggers multiple signals.
Take a representative case from the second assignment, which asks students to implement a function that returns the average of a list of numbers. Two submissions looked like this.
# Student A, submitted Monday
def compute_average(nums):
total = 0
for n in nums:
total += n
return total / len(nums)# Student B, submitted Thursday
def calculate_mean(values):
sum_values = 0
for value in values:
sum_values = sum_values + value
return sum_values / len(values)These two functions are not textually identical. Variable names differ, the function name differs, and the loop accumulation uses a different syntactic form. A surface string match would miss them. A token-based checker would catch some overlap because the token sequence still shares structural elements. But Codequiry's AST comparison sees the underlying control flow and data flow as the same pattern. The tool flagged the pair at 91% similarity, and the side-by-side view made the structural equivalence legible to a human reviewer.
What MOSS would have done with this pair depends on the threshold. At a typical threshold of 60% or so, this pair may or may not have crossed. The department had been using a threshold tuned for longer assignments. For short functions, token overlap is noisy. The layered approach added a second signal: Codequiry also checked the open web and found a near-identical function in a public tutorial repository, which suggested both students may have drawn from the same online source, or one copied from the other who had copied from the web. The web match gave the instructor context that peer similarity alone never could.
Then the department turned on the AI check for the same assignment. The AI detector reported that Student B's file had an 87% probability of being generated by an LLM, while Student A's file scored 34%, which is below the threshold the department chose for further review. The instructor opened the AI report and saw the per-file breakdown, with the highest-confidence file listed first.
This layered result changed the conversation with Student B. Instead of starting from "you copied from a classmate," which the student could deny, the instructor could say, "your submission shows three independent signals: structural similarity to another submission, a matching web source, and a high AI-generation probability. Let's talk about how you produced this code." That framing is not accusatory in the same way. It gives the student room to explain, and it gives the instructor a documented basis for whatever follows.
The Investigation Workflow for Flagged Submissions
The department did not want to turn every flagged submission into an automatic honor-code case. That would be unfair to students who legitimately used similar patterns or who used AI assistance within the course's stated policy. So they built a review queue, not a verdict pipeline.
The process worked in three stages. First, every submission above peer similarity threshold or web match threshold or AI probability threshold entered the queue. Second, the head TA reviewed the top entries using the side-by-side comparison and web source list. Third, only the cases that survived that review went to the instructor for a conversation. The Codequiry dashboard made this manageable because it ranked submissions by an outlier score rather than requiring a manual scan of all 412 files.
One detail we all forgot at first: the language identifier. The first run on the third assignment, which was in C++, had been configured with Java as the language because the course had switched from Java to C++ the previous year and the check template still defaulted to Java. That caused a batch of false positives on the first night, roughly twenty submissions flagged as high-risk that were actually fine. We caught it the next morning when a student asked why a simple loop was marked 100% similar to something. The fix took ten minutes. The lesson was that detection workflows need configuration review just like any other teaching tool, especially when languages change between semesters.
I will hedge here: we have not tested this layered workflow beyond the roughly 800 submissions across the two semesters of Intro to Computer Science I. The results were consistent, but I would not want to generalize to a 2,000-student data structures course without recalibrating thresholds.
Tradeoffs and What the Department Learned
The biggest tradeoff was time. Layered detection produced more flags than MOSS alone, which meant more human review time, not less. The department's initial fear was that this would overwhelm the teaching staff. In practice, the smart review queue helped, but the head TA still spent about three to four hours per week during assignment grading weeks reviewing flagged cases, up from about one hour under the old MOSS-only workflow. The upside was that the flagged cases were more substantive, and several of the AI-flag cases would never have surfaced otherwise.
A second tradeoff involved policy. The department had not updated its academic integrity statement to address AI-generated code. When the first AI flags appeared, the instructor had no clear rule to apply. Was submitting an LLM-generated solution without attribution plagiarism? The department's honor code said plagiarism meant using another person's work without citation. An LLM is not a person, technically. The department spent a full faculty meeting debating this, and the eventual policy distinguished between AI-assisted work (using a tool to debug, explain, or suggest small fragments) and AI-generated work (submitting code that the student did not substantially design). The layered detector could not make that distinction, but it provided the evidence that made the conversation possible.
Detection tools do not resolve ambiguity; they reveal it. The educational value is in what happens after the flag, not before.
That quote came from the chair, and it has stayed with me. A detector that only told you "student X copied from student Y" would have let the department continue pretending the only integrity problem was peer copying. A detector that surfaced web matches and AI probabilities forced the department to articulate what they actually expected from students.
Where Codequiry Fit Into the Stack
The department had used MOSS for years, and I have no interest in dismissing it. MOSS is fast, free, and familiar. What it lacks is the combined signal: peer similarity, web source tracing, and AI-generation probability in one report, with a usable interface for instructors who are not tooling experts. Codequiry's source code plagiarism checker fills that gap without requiring the department to run three separate services and merge the results manually.
One practical point for colleagues considering this move: the API mattered. The department's autograder already collected submissions and ran them through a series of unit tests. Adding a Codequiry API call to the grading pipeline meant that similarity, web, and AI checks could run as part of the same submission event, not as a separate after-the-fact batch. That reduced the temptation to skip detection because "we already graded everything." The REST API accepted the same file the autograder received, and the results came back indexed by submission ID, which the course management system could attach to the student record without extra manual work.
I want to be clear about what a detector cannot do. It cannot determine intent. It cannot tell you whether a student who used an LLM to generate a first draft and then spent two hours modifying it has violated the spirit of the course. That judgment belongs to instructors. The detector gives you a signal, not a verdict. The department learned this the hard way when a student with a high AI probability score turned out to have written the code herself but had used an AI code completion tool that suggested the exact function she was about to write anyway. Her explanation was credible, and the case was dropped. The department then adjusted its policy to require disclosure of AI code completion tools in the submission comments, which turned out to be more productive than trying to police every keystroke.
What This Means for CS Departments Considering a Similar Move
If you are a CS professor or department chair wondering whether to adopt layered detection, the honest answer is that it depends on your scale and your current pain. For a 20-student seminar, a careful TA can eyeball every submission and know what is off. For a 400-student intro course with seven graders, eyeballing is not reliable, and a single similarity tool leaves blind spots. The department in this case found that the combination of peer, web, and AI checks in one dashboard reduced the number of cases that slipped through to the honor board while also reducing false accusations, because the evidence was more complete before a conversation began.
You will need to budget for review time and policy revision. You will need to configure language identifiers correctly, as I mentioned. You will need to decide what to do with borderline AI probability scores, because no detector is perfect. But you will also see patterns you were missing before, and you will have better conversations with students about what original work means in a programming course where code reuse is both normal and sometimes inappropriate.
For the department I observed, the move to layered detection did not end academic integrity cases. It made them more legible, more defensible, and more educational. That seems like the right goal for a teaching institution. If you are exploring a similar change, the AI code detector integrated with peer and web checks is a reasonable place to start, because you can run it on a single assignment before committing to a full semester workflow.