Midwestern State University ran MOSS on every programming assignment for a decade. The tool flagged near-identical submissions, the teaching assistants matched them to honor-code hearings, and that was that. Then, in the fall of 2023, the chair of the CS department noticed something unsettling: the same students whose MOSS score was a clean 0% were turning in code that read like a textbook solution from a slightly different universe—over-commented, hyper-consistent, with variable names that didn’t quite match the student’s usual style.
They weren’t plagiarizing. They were generating.
That realization kicked off a six-month experiment in what the department now calls “layered detection”—running a traditional code plagiarism checker on every submission, then following up with a dedicated AI detector on the ones that cleared the similarity threshold. The results changed how the department thinks about academic integrity.
The Old Way: MOSS Alone
Like most CS departments, MSU relied on Stanford’s MOSS (Measure of Software Similarity) for years. MOSS is fast, free, and remarkably good at catching copy-paste plagiarism and light refactoring. It tokenizes code and uses winnowing to generate fingerprints, then compares submission pairs. Students who rename variables or reorder functions still trigger a match because MOSS operates on structural fingerprints, not line-by-line diff.
The TAs liked it. They’d set a similarity threshold of 60% for most assignments, manually inspect anything above that, and flag the clear-cut cases—students who shared code verbatim, copied from a single GitHub repository, or pasted from a tutorial site. The system handled 80% of academic integrity cases cleanly. The remaining 20% were ambiguous: pairs around 40–55% similarity that might be legitimate collaboration or independent convergence on a simple algorithm.
But MOSS has a fundamental limitation: it only compares submissions against other submissions or against a supplied base-code corpus. It won’t flag a student who pasted code from an un-indexed forum, used a private LLM, or carefully hand-typed an AI-generated solution while deliberately varying the structure. It looks for copied code—not originated code that happens to be suspiciously uniform.
The AI Wave Hits
By early 2023, CS instructors everywhere were noticing patterns they couldn’t unsee. Assignments that should have taken a struggling student hours were coming back in 40 minutes. Code showed an eerie level of consistency: every variable declared at the top of the function, every conditional block followed by a explanatory comment, every edge case handled in exactly the same order. The logic worked, but the authorship didn’t feel right.
MSU’s department decided to run a controlled test. They took 200 student submissions from a data structures course—100 from 2021 (pre-ChatGPT) and 100 from 2023—and ran them through two separate detection pipelines. First, the standard MOSS check. Then, a dedicated AI code detector trained to recognize telltale statistical signatures of LLM-generated code: low perplexity (predictable token sequences), uniform whitespace patterns, and consistent identifier naming conventions that differ from a human’s natural variability.
The results were sobering. MOSS flagged 11% of the 2023 submissions as suspicious—roughly the same rate as 2021. But the AI detector separately flagged an additional 18% of submissions with no MOSS matches as having high probability of AI generation. Some of those were likely false positives. But manual review confirmed that at least two-thirds of them exhibited unmistakable LLM patterns: the distinctive “solve the problem three different ways in comments” output, perfectly aligned docstrings on every function, and code that was too idiomatic for a sophomore who struggled with pointer syntax two weeks prior.
MOSS alone was missing nearly a third of the actual integrity issues.
Deciding to Layer Checks
The department considered several approaches before settling on a two-stage pipeline.
Option one: replace MOSS entirely with a combined tool. But MOSS is deeply embedded in their grading workflow, and TAs are trained on its similarity reports. Ripping it out wasn’t practical—and MOSS still caught the classic copy-paste cases better than any AI detector could, because AI detection looks for statistical patterns, not identical structure.
Option two: run AI detection on every submission first. This flopped in testing because an AI detector alone produces a higher false-positive rate than instructors were willing to tolerate. Students who write unusually clean, idiomatic code—often the top performers—triggered false positives. TAs ended up doing more manual review, not less.
The third option, and the one they ultimately deployed, was to run similarity checks first, then route only the cleared submissions—those below the similarity threshold—through the AI detector. The logic: if a student already triggered a MOSS flag, they had a plagiarism problem and the case could proceed on those grounds regardless of AI involvement. The AI detector was reserved for catching the generation-only cases that similarity tools miss entirely.
This separation reduced the AI detector’s review volume by roughly 70% (since most submissions cleared similarity) and kept the overall false-positive workload manageable. TAs weren’t drowning in AI flags; they were only looking at the subset of clean-by-similarity submissions that still seemed off.
How the Two-Stage Pipeline Works
Here’s the concrete workflow MSU implemented for a typical 120-student data structures course:
- Submission collection: Students submit via the LMS. All submissions are archived with timestamps.
- Stage 1 — Similarity scan: Every submission is run through MOSS (or, for instructors who want a richer report with web-source matching, a source code plagiarism checker that also checks against GitHub and Stack Overflow). Submissions with pairwise similarity above 50% are flagged for manual review. Those with matches to known web sources are flagged separately.
- Split the pool: Flagged submissions go to the existing plagiarism review queue. Cleared submissions (similarity under 50%, no web matches) advance to Stage 2.
- Stage 2 — AI detection: Each cleared submission is run through an AI detector that performs per-file analysis, scoring the probability that the code is LLM-generated. The department set a threshold of 75% confidence before placing a submission in the AI-review queue.
- Manual review queue: TAs review both queues separately. Plagiarism cases follow the standard honor-code procedure. AI-flagged cases get a lighter touch: the TA looks for corroborating signals like timestamp patterns, style mismatch with the student’s earlier work, or over-commenting that matches known LLM patterns.
Codequiry’s platform became the backbone for the combined pipeline because it handles both stages without requiring separate tools. The similarity engine uses token-based and AST fingerprinting that catches refactoring attempts—students who rename every identifier and reorder methods still match. The AI detection module runs on the same submission set and surfaces per-file scores across four LLM families, including ChatGPT, Copilot, and Claude.
What They Found
After running the layered pipeline on three assignments across two semesters, the department published internal numbers. (The following data is from their fall 2023 and spring 2024 cohorts, totaling 720 submissions.)
| Detection Method | Submissions Flagged | Confirmed Violations | False Positive Rate |
|---|---|---|---|
| MOSS only (≥50% similarity) | 62 (8.6%) | 58 | 6.5% |
| AI detector only (≥75% confidence) | 97 (13.5%) | 74 | 23.7% |
| Layered: MOSS first, then AI on remainder | 109 total (15.1%) | 101 | 7.3% overall |
The combined approach caught 43 additional confirmed violations that MOSS alone missed—an increase of 74% in detected integrity cases—while keeping the false-positive rate nearly identical to MOSS alone. The AI detector’s standalone false-positive rate was high, but because it only ran on the subset that already passed similarity, the absolute number of false flags requiring review was minimal.
One notable finding: many of the AI-flagged submissions came from students who had real plagiarism flags on earlier assignments. A student who copied code in week 2 was also likely to generate it in week 6. The overlap suggested that AI generation wasn’t replacing plagiarism; it was supplementing it. Students who intended to cheat used whatever tool was available.
Handling False Positives with AI Detection
The 23.7% false-positive rate for AI detection alone is the reason MSU never considered running it as the primary check. When you flag 97 submissions and 23 are legitimate student work, you create a trust problem. Students accused of AI use often react more defensively than those accused of copying—because copying from a peer feels like stealing, while generating code can feel like using a calculator. The emotional stakes are different.
MSU’s policy for AI-flagged cases evolved over the semester. Rather than immediate honor-code proceedings, flagged submissions triggered a “style interview.” The TA sat down with the student and asked them to explain their code, modify a function on the fly, or walk through the logic of a particular loop. Students who actually wrote the code could do this easily, even if their style triggered the detector. Students who couldn’t explain a variable declaration they “wrote” two days earlier had a harder time.
This interview process turned the AI detector from a prosecutorial tool into a screening tool. The false-positive rate dropped to near zero in terms of actual penalties, because the interview served as the real determinant.
Why a Combined Platform Matters
The friction in MSU’s early pipeline was tool-switching. TAs had to export MOSS results, filter them, then upload the cleared subset to a separate AI detector, then merge the two report sets for each student. It worked, but it was brittle. When the department evaluated platforms that handled both similarity and AI detection natively, the workflow collapsed into a single dashboard: upload once, see both flags side-by-side, and sort the review queue accordingly.
Codequiry’s approach to layered detection is built around this exact pattern. Because the similarity engine and AI detector share the same submission index, a TA can open a student’s report and see simultaneously: which peers their code matched, which web sources they may have copied, and what probability score the AI detector assigned. That single-pane view eliminated the tool-switching overhead and let TAs make faster, more confident decisions.
For departments still using MOSS, there’s also a practical comparison worth noting. MOSS is excellent at pairwise similarity, but it doesn’t check the open web, and it provides no AI detection whatsoever. Running a MOSS alternative that adds web-source scanning and AI detection can fill those gaps without abandoning the similarity-first workflow that TAs already trust.
The Student Reaction
When MSU announced the new policy—layered detection, AI interviews—some students pushed back. A few argued that using ChatGPT to generate boilerplate code was no different from using an IDE’s autocomplete. The department’s response was pragmatic: boilerplate is fine if declared, but submitting an entire assignment where the logic, structure, and comments were generated by an LLM without attribution crossed the line.
They updated the syllabus to include a specific AI-use policy distinguishing between “assistive” and “generative” use. Assistive: using Copilot to autocomplete a function body the student is actively designing. Generative: pasting the entire assignment prompt into ChatGPT and submitting the output. The line isn’t always bright, but the interview process clarified it quickly. Students who could demonstrate understanding kept their grade. Those who couldn’t received a zero and a conversation about academic integrity.
What Other Departments Are Doing
MSU isn’t alone. A growing number of CS programs—from large R1 universities to community college coding bootcamps—are implementing similar layered detection pipelines. The common thread is a recognition that no single detection method is sufficient anymore. Code similarity caught the copy-pasters. AI detectors catch the generators. Together, they cover the majority of integrity threats.
Some departments have gone further, integrating code plagiarism checker for teachers tools directly into their LMS via LTI or API, so that similarity and AI checks run automatically on submission without TA intervention. The results are surfaced in the grading interface, and only flagged submissions require human review. This reduces the TA workload to a manageable handful per assignment rather than dozens.
The Tradeoffs That Remain
Layered detection isn’t perfect. AI detectors still produce false positives on students who write unnaturally clean code—often the ones who’ve internalized style guides to the point of removing all human idiosyncrasy. Similarly, web-source scanners can flag students who legitimately reused their own code from a public repository. The process requires thoughtful thresholds and a human in the loop.
But the alternative—running only similarity checks and watching a wave of generation go undetected—is increasingly untenable. Every semester that passes without AI detection means more students learn that they can bypass integrity checks simply by not copying from a peer. The normalization of undetected AI use erodes assignment validity faster than any single plagiarism case.
Frequently Asked Questions
Why run AI detection after plagiarism detection instead of before? Plagiarism detection has a lower false-positive rate and catches straightforward copy-paste cases. Running it first removes those cases from the pool, so the AI detector only analyzes submissions that are structurally unique—reducing the total volume and the absolute number of false positives. What threshold should I use for AI detection? Most departments start with a confidence threshold between 70–80%. Below 70% produces too many false positives; above 80% may miss nuanced AI-generated code that has been lightly edited. The right threshold depends on your tolerance for manual review, but 75% is a common starting point. Does MOSS detect AI-generated code? No. MOSS detects structural similarity between submissions using fingerprinting, but if a student generates original code with an LLM and does not copy it from another student, MOSS will find no matches. AI-generated code can be entirely unique and still pass MOSS with a 0% similarity score. How do you handle a student who claims they wrote AI-flagged code? A short style interview—asking the student to explain their logic, modify a function, or trace through a specific block—quickly distinguishes genuine authorship from AI generation. Most students who genuinely wrote the code can explain it fluently, even if the statistical signature triggered the detector.Layered detection is becoming the default integrity architecture in CS education, not because any single tool failed, but because the threat surface expanded. When you run only similarity, you miss generation. When you run only AI detection, you drown in false positives. Together, with careful thresholds and a human review layer, they catch what neither can alone. See how Codequiry’s combined platform handles both checks in a single pipeline.