I mark 320 programming assignments every week during the first half of the semester. By the time the midterm project rolls around—usually a text-based adventure in Java or a simple shell in C—I’m already exhausted. The thing that used to drain me most wasn’t the grading. It was the hours of squinting at MOSS output, trying to decide if six students with suspiciously identical readLine() wrappers had actually collaborated, or if my starter code had naturally funneled them toward the same solution.
After three semesters of refining the approach, I can now triage the raw similarity reports for a 300‑student section in about 90 minutes, flag only the cases that merit a follow‑up conversation, and file honor‑code referrals that hold up under scrutiny. Here’s what changed.
The First Semester Was Naive
Like most instructors, I started with a single MOSS run on the submission directory. I’d upload everything, wait for the email, and then wade through a raw HTML list of pairs ranked by percentage match. The top 100 pairs all looked the same to my tired eyes—variable names shuffled, comments stripped, for loops turned into while loops. I spent six hours chasing false positives. The worst part was the human cost: students who had genuinely done the work alone felt accused when I asked them to explain a 72% match that turned out to be boilerplate I had provided.
I needed to stop treating the raw % similarity number as a truth value and start thinking in layers. The core question was never “How similar is this code?” It was always “Where does this similarity come from?”
Layering the Checks
By the second semester, I’d settled on a four‑layer protocol. Whenever a pair of submissions crossed a threshold, I asked:
- Is the overlap concentrated in starter code or standard library wrappers?
- Could the match be explained by a shared online source—Stack Overflow, a tutorial, a public GitHub repo?
- Does the structural evidence survive renaming, reordering, and refactoring moves that a student would actually make?
- If I read both files side‑by‑side, can I articulate what specific creative choices were duplicated?
Layers one and two eliminated roughly 60% of flagged pairs immediately. For scraping‑style plagiarism from the web, MOSS gave me nothing—it only compares against the submission pool. That’s where a source code plagiarism checker with web‑corpus scanning turned out to be indispensable. I’d run the top matches through a tool that checked against GitHub, Stack Overflow, and known tutorial sites. Within minutes, I could see that a student’s suspiciously elegant mergeSort implementation was actually a verbatim paste from Rosetta Code. That changed the conversation from “Did you copy from a classmate?” to “Did you attribute this external source?”—a much cleaner, less adversarial line.
Where Token‑Based Tools Missed the Mark
MOSS’s winnowing algorithm is clever—it breaks code into k‑grams of tokens and fingerprints them. But when an upstream student shares an answer and a downstream student systematically changes every variable name, reorders methods, and inlines helper functions, token‑level similarity drops into the high 60s, which in a class of 300 blends into the noise. I needed an AST‑aware check that could say “the control‑flow graph of these two solutions is identical despite the surface text looking different.”
I trialed a few commercial platforms that claimed AST‑based comparison. Most returned opaque “match scores” without letting me see why. Codequiry’s similarity report gave me a side‑by‑side view that highlighted block‑level correspondences—function bodies that were structurally identical even though every identifier had been altered. That let me check whether the logic genuinely represented independent learners converging on the same algorithm, or whether someone had taken a full solution and ran a refactoring script. One case I’ll never forget: two students submitted code for a maze solver where the only difference was that one had renamed x to col and y to row throughout. Token similarity was 82%. The AST fingerprint was 99.2%. I showed one student the structural comparison and got a quiet confession in under two minutes.
The 90‑Minute Triage Protocol, Step by Step
Here’s the exact sequence I run now, using a combination of MOSS (for the preliminary peer pool scan) and Codequiry (for web‑source checks and structural fingerprinting).
Step 1: Generate the peer‑similarity matrix. I upload all submissions to MOSS and set the sensitivity to -m 5 to filter out very short common sequences. I also run the same batch through Codequiry’s peer‑comparison module, because having two independent matching engines reduces blind spots.
Step 2: Exclude starter‑code noise. I maintain a “known common” file that contains all provided skeleton code—the class definition, method stubs, import statements. Any pair where the overlapping lines are 80% contained in that file is dismissed instantly. In my CS1 course, this cuts the report size by nearly a third.
Step 3: Run web‑origin checks on high‑similarity clusters. I take every pair with MOSS similarity above 65% or Codequiry similarity above 75%, and check them against online sources. If a match appears on GeeksforGeeks or a public GitHub repo, I note it and move on. If the pair is clean of web matches but still structurally identical, it goes to the next tier.
Step 4: Spot‑check structural fingerprints. This is where I focus my remaining attention. For pairs that survived steps 2 and 3, I open the side‑by‑side AST comparison. I look for identical control‑flow between non‑trivial functions—loops with the same unusual exit condition, helper methods called in the same peculiar order, if‑else chains with the same string literals. If I can’t explain the overlap without invoking copying, the case moves to the “ask” pile.
Step 5: The human read. I physically read both files for every “ask” case. I’m not looking for similarity anymore; I’m looking for the marker of an independent mind: a strange comment, a half‑finished refactor, an idiosyncratic variable name like bananaCounter. If I find one in both files, that’s usually the nail. If I don’t, I write a short email with screenshots and ask the student to walk me through their implementation.
This sequence, run Wednesday evening after the Tuesday deadline, now takes me between 80 and 110 minutes for ~300 submissions. I flag about 12 pairs for follow‑up, and on average 7 of those result in a confirmed violation. The false‑positive rate on flagged cases has dropped to nearly zero, which matters enormously when you’re a TA who has to face those students in lab the next day.
What I Learned About Designing Plagiarism‑Resistant Assignments
No amount of detection can compensate for an assignment that invites collusion. In my first semester, I gave a “Bank Account” project that had exactly one sensible object‑oriented decomposition—a parent Account class with CheckingAccount and SavingsAccount subclasses. Half the class produced structurally identical designs. It wasn’t plagiarism; it was a narrow design space.
Now I design for “divergent surface, convergent semantics.” Every project has at least three places where students must make an aesthetic choice that doesn’t affect correctness: naming conventions for user‑facing strings, the order of output formatting, whether to use recursion or iteration for a tree traversal that works either way. When two submissions match on all those aesthetic markers, it’s far more telling than a matching for loop. I also periodically change the assignment’s thematic wrapper—the same algorithm dressed as a weather simulator one semester and a stock ticker the next. That makes it harder for fraternity file caches to stay useful.
Why a Dashboard Matters for Instructor Sanity
I’m a TA first, researcher second. I don’t have time to script a custom pipeline that stitches together MOSS, JPlag, and a web scraper. The single biggest productivity gain came when I adopted a code plagiarism checker for teachers that put peer matches, web matches, and AI‑generated flags into one unified report, sortable by severity, and accessible from a browser. I log in, pick the assignment, and get a prioritized list of cases instead of a 3,000‑line HTML table. Even if the underlying algorithms were identical to open‑source MOSS, the UI alone saves me an hour a week.
When I began to suspect that a handful of students were using ChatGPT for the more open‑ended assignments, the same dashboard gave me an AI‑detection column without my having to sign up for yet another service. The AI‑written code detector flagged submissions with GPT‑2‑level perplexity scores below a threshold, alongside the traditional similarity checks. No tool should accuse a student based on a single AI score alone, but the layering principle applies again: a submission that hits 97% peer similarity and an AI‑like perplexity score and matches a GitHub source in the web module is not ambiguous.
Concrete Numbers from Three Semesters
| Semester 1 | Semester 2 | Semester 3 | |
|---|---|---|---|
| Submissions | 289 | 312 | 335 |
| Raw MOSS pairs above 70% | 142 | 155 | 168 |
| After starter‑code exclusion | 98 | 106 | 114 |
| After web‑source check | 51 | 44 | 41 |
| After AST fingerprint review | 18 | 15 | 12 |
| Confirmed collusion cases | 11 | 9 | 8 |
| Review time (minutes) | 360 | 140 | 90 |
The drop in confirmed cases isn’t a sign that the detector is weakening; it’s a reflection of the fact that students know I’m checking carefully. Word travels fast through a department when the first few project reports result in formal honor‑code violations. The deterrence effect is real, and it’s the ultimate goal of any integrity system.
When to Err on the Side of Caution
I still catch myself wanting to flag edge cases out of sheer frustration. Two submissions that share a single helper function, nothing else—maybe they studied together. A few identical comments because the lab handout used that phrasing. A match on a public static void main that’s literally the first example in the textbook.
The rule I give new TAs: if you can’t write a two‑sentence narrative explaining why a human being would have needed to see the other person’s code to produce this submission, don’t flag it. The tool provides evidence, not a verdict. That’s the difference between a forensic analysis that holds up and a fishing expedition that burns trust.
Where the Tooling is Headed
The next thing I want is continuous integration for academic integrity—every commit to a GitHub Classroom repository gets checked against the cohort, the web, and an AI‑likeness score before the deadline, with a gentle warning nudging the student to review their own work’s originality. No accusations, just “hey, your code is 94% similar to a classmate’s; you might want to talk to your instructor before submitting.” Platforms like Codequiry already have a REST API that makes this technically feasible. When I tested a prototype with a small honors section last fall, the number of cases I had to investigate post‑deadline dropped to two. One of those turned out to be a student who’d committed the other person’s file by accident—a teachable moment, not a violation.
Frequently Asked Questions
How do you handle false positives from MOSS?
I cross‑reference every high‑similarity pair against a starter‑code exclusion list, then scan for web sources, and finally review structural fingerprints. Routinely, 60% of initial MOSS flags are dismissed in the first two layers. I never accuse a student solely on a MOSS percentage.
What’s the difference between token similarity and AST fingerprinting?
Token similarity compares sequences of lexical elements after normalization; it’s sensitive to renaming but can miss refactoring that rearranges code blocks. AST fingerprinting builds a structural hash of the abstract syntax tree, so reordered functions and renamed identifiers don’t affect the match. That catches the cases where a student runs a script to mechanically alter the surface.
Can web‑source checks detect code copied from private GitHub repos?
Publicly indexed repos, yes. Private repos are not scanned by any legitimate service; that would require credentials and raise ethics concerns. For private repository sharing, peer‑comparison within the submission pool is still the most effective detection method.
How long does it take to learn this triage workflow?
Most TAs become proficient after two assignment cycles. The key is having a layered checklist and a dashboard that consolidates the evidence. We train new staff by having them shadow an experienced reviewer for one week, and they’re flying solo by the first project deadline.
If you’re staring at a mountain of similarity reports and wondering whether there’s a sustainable way through it, there is—and it starts with layering the right signals. Try a code plagiarism checker that gives you web, peer, structural, and AI‑generated flags in a single view.