Every January I run the same experiment on myself. Two hundred and ninety submissions arrive for the first data structures assignment, and before I open any of them I pick twenty at random and guess which ones will surface in a scan. My hit rate is about sixty percent. That is not a knock on my instincts; it is a reminder that the scan sees populations I do not, and that my guesses about which students would cheat are worth less than a sorted list.
Here is the short version of what follows. You scan a course for copied and AI-written code by treating it as three separate questions, peer similarity, web and repository provenance, and machine generation, each with its own pipeline and threshold. Then you merge the three into a single ranked queue and read the top of it with human eyes. Everything below is the detail behind that sentence.
Step 1. Separate the three questions before you buy or configure anything
Peer similarity compares submissions in your cohort against each other. If two students in the same section turn in the same forty lines, that is a peer match, and no external database is involved.
Web and repository provenance compares a submission against code published elsewhere: GitHub, Stack Overflow answers, tutorial sites, pastebins that search engines have indexed. This catches a different population than peer similarity does, and I have seen departments buy a peer-only tool and then wonder why half their cases walked through it.
Machine generation is a probability estimate, per file, that a language model produced the text. It is the newest of the three and the one most often misconfigured, because people run it once per submission when they should run it once per file.
These pipelines overlap less than you would expect. A student who pastes a public solution from a GitHub repository of the same assignment will score near zero on peer similarity and very high on web. A student who asks a model to write the assignment from scratch will often score low on both, because the output is not published anywhere, and only the AI pipeline will move. If you run one engine, you are running one third of a scan.
Step 2. Clean the corpus, because your first scan is mostly noise
Before any engine touches the files, build a manifest. LMS exports hand you folders named submission_8471, zip files nested inside zip files, and at least one submission that contains an entire Eclipse workspace with a .metadata directory in it. Most scanners will happily ingest all of that and report similarity between two identical .classpath files.
I normalize every source file and compute a hash of the normalized text. Files that hash identically are exact copies, and they deserve a different, cheaper conversation than a full similarity report.
import hashlib, pathlib, re, csv
def normalize(src: str) -> str:
src = re.sub(r"/\*.*?\*/", "", src, flags=re.S) # block comments
src = re.sub(r"//[^\n]*", "", src) # line comments
return re.sub(r"\s+", "", src).lower() # whitespace and case
rows = []
for path in pathlib.Path("submissions").rglob("*.java"):
raw = path.read_text(encoding="utf-8", errors="replace")
norm = normalize(raw)
rows.append({
"student": path.parts[1],
"file": path.name,
"lines": raw.count("\n"),
"sha256": hashlib.sha256(norm.encode()).hexdigest()[:16],
})
with open("manifest.csv", "w", newline="") as f:
csv.DictWriter(f, fieldnames=rows[0].keys()).writerows(rows)
Two things come out of this table. Group by sha256 and you have your byte-identical pairs, which are the easiest cases you will see all term. Filter on lines < 40 and you have a list of files too short to support a meaningful similarity judgment, which is most of what a bad scan reports as a false alarm.
I learned the second half of this step the hard way. One spring I forgot to exclude the edu.course2100.starter package that I hand out with the assignment. The scaffold includes a Node class and a small I/O helper. Every student shipped it unmodified, and the whole cohort lit up between 38 and 44 percent similarity. I spent a Saturday morning reading reports before I realized I was looking at my own code.
Add your starter files to the exclusion list. Add them every semester. They are the single most common source of wasted review hours in this work.
Step 3. Run the peer scan with an explicit base-code exclusion
The classic tool here is MOSS, written by Alex Aiken at Stanford and in continuous use since 1994. It is free, it is command-line, and it is based on winnowing, which is the same document-fingerprinting family of algorithms that JPlag and Dolos build on. The winnowing idea in brief: break the code into overlapping chunks of length k, hash each chunk, and keep only the minimum hash in each sliding window of size w. What remains is a small, refactoring-tolerant fingerprint of the file.
def kgrams(tokens, k=9):
for i in range(len(tokens) - k + 1):
yield hash(tuple(tokens[i:i + k]))
def winnow(hashes, w=4):
kept = set()
for i in range(len(hashes) - w + 1):
kept.add(min(hashes[i:i + w]))
return kept
Because a fingerprint is a set of hashes rather than a string, comments, whitespace, and variable renaming mostly fall away. That is why renaming i to index across a submission accomplishes so little. Structural changes are the ones that actually move the number.
For MOSS itself the invocation looks like this:
./moss -l java -m 10 -b starter/Node.java -b starter/io/Reader.java submissions/*.java
The -m flag sets the minimum match length in lines and is the knob you will actually tune. Ten lines is a reasonable default for a 300-level assignment; for a two-hundred-line introductory exercise, drop it to six and expect more noise. The -b flags name base files or directories to ignore. Results come back as a URL in an email, with no dashboard, no history of your previous runs, and no per-student record you can hand to a committee without copy-pasting.
JPlag, from KIT, gives you a local HTML report instead of a URL, and version 5 has a considerably better viewer than the 4.x line I used through 2021. Dolos, from the Helmholtz Center for Information Security, is a Node/TypeScript tool with a browser interface and supports around ten languages. All three are legitimate. If your department has a MOSS habit dating back fifteen years, it will keep working.
What a hosted source code plagiarism checker adds is not a better algorithm, it is an artifact: a persistent report per semester, per course, per check, that a colleague can open in July without asking you to re-run anything. Codequiry keeps the same peer, web, and AI scores on one submission record, which matters when a committee asks why a specific student was flagged.
Step 4. Run the web and repository check separately, and know its blind spots
The web pass compares submissions against indexed code: GitHub public repositories, Stack Overflow answer blocks, tutorial sites, course pages that have leaked into the open. It catches the student who found a repo for exactly this assignment, and it catches the student who pasted a Stack Overflow helper without attribution.
It has real blind spots, and you should be able to state them in a committee meeting. Code in a private repository is invisible. Code in a paywalled course or a Discord study group is invisible. Time spent on Chegg is invisible unless the posting surfaced publicly. And a model's output is invisible by definition until someone publishes it.
The web pass also gives you something the peer pass cannot: a provenance record. When a submission matches three lines of a Stack Overflow answer with the variable names changed, that is not necessarily misconduct. It is often a student doing what the site encourages. Having the match, with the line and token counts attached, lets you have that conversation honestly instead of guessing.
Step 5. Run AI detection per file, never per submission
Two terms first, since they get thrown around loosely. Perplexity measures how surprised a language model is by a piece of text, and low perplexity means the text is highly predictable to the model. Burstiness measures variation in that predictability across the document. Human writing, and human code, tends to move around: dense here, loose there, an odd comment in the middle of a loop, a helper method that exists only because the author got stuck.
Adapting these to source code takes more than running a text model over a .java file. The signals that carry weight are structural: uniform docstring coverage on private helpers, defensive validation the assignment never requested, complete error handling on a path that cannot fail, and an absence of the debugging residue that real student code almost always has. Commented-out lines. A TODO from Tuesday. A variable named temp2 that exists because the first temp was already taken.
Here is the kind of comparison that draws my eye. A student's merge helper, submitted in week six:
// merges two sorted halves
static void merge(int[] a, int lo, int mid, int hi) {
int[] tmp = new int[hi - lo + 1];
int i = lo, j = mid + 1, k = 0;
while (i <= mid && j <= hi) {
if (a[i] <= a[j]) tmp[k++] = a[i++];
else tmp[k++] = a[j++];
}
while (i <= mid) tmp[k++] = a[i++];
while (j <= hi) tmp[k++] = a[j++];
for (int x = 0; x < tmp.length; x++) a[lo + x] = tmp[x];
}
And a version with a different signature:
/**
* Merges two sorted subarrays of the given array in place.
*
* @param array the array containing both subarrays
* @param left the starting index of the first subarray
* @param middle the ending index of the first subarray
* @param right the ending index of the second subarray
* @throws IllegalArgumentException if the indices are out of range
*/
private static void merge(int[] array, int left, int middle, int right) {
if (left < 0 || middle >= right || right >= array.length) {
throw new IllegalArgumentException("Invalid merge bounds");
}
int[] temp = Arrays.copyOfRange(array, left, right + 1);
...
The second version is not evidence of anything by itself. But the Javadoc on a private helper, the bounds check the assignment never asked for, and the preference for Arrays.copyOfRange over a manual copy loop are the pattern I have come to associate with generated code, and they are also exactly the pattern an edit-heavy student will erase when they rewrite it.
That is why you scan per file. A submission with two of nine files scoring near 0.9 is a very different case from one with nine of nine files near 0.4, and a submission-level average flattens both into a number with no useful shape. A good AI code detector will show you the per-file distribution and the written indicators behind each score, so your conversation with the student can be about a specific file rather than about an accusation in general.
I will be honest about the limits here. Detectors get weaker on short files, on standard boilerplate, and on code the student rewrote substantially after generating it. In my experience the per-file scores are strongest on files between roughly 40 and 400 lines that contain an actual algorithm. A 12-line accessor class tells you nothing, and no tool can make it tell you something.
Step 6. Triage the union, ranked, and give each case a time budget
You now have three lists. The mistake I made for two semesters was reading them in alphabetical order, which is the slowest possible way to find the twenty cases that matter.
Rank instead. Sort by an outlier score relative to the cohort, take the top of the queue, and give each case a hard twenty minutes. Most cases resolve in five. A match above about 85 percent on multiple files with matching structure across a rename is not a subtle question. A 40 percent match on a shared utility helper is, and it usually resolves to nothing.
The reason to work from a queue rather than from suspicion is practical and I say it plainly to my TAs: we flag what the numbers flag, in the order the numbers rank them, because a review process that depends on whose name looks familiar is not a process. It is also the only version of this that survives an appeal.
Step 7. Build the evidence packet before you contact anyone
By the time you email a student, you should have a single document that a colleague could read cold. Over the years I have settled on the same contents:
- The submission as received, with the timestamp from the LMS export.
- The matched source, whether that is a peer submission, a URL with a capture date, or the per-file AI score with its indicators.
- A side-by-side diff, not two scrolls of code. The committee reads diffs.
- Your own note on which structural elements match: the loop shape, the off-by-one handling, the same unusual variable ordering.
- The course policy as it read on the assignment's due date, not the version you revised afterward.
That fourth item is the one people skip and the one that decides cases. Renamed variables and reformatted code are not a defense when the structure is identical line for line, and the diff makes that visible in ten seconds.
Step 8. Calibrate at the end of the semester, or the thresholds will drift on you
Record four numbers each term: how many submissions the scan flagged, how many you reviewed, how many the student acknowledged, and how many you cleared. I keep mine in the same spreadsheet going back to 2016.
Two patterns have held steady. Roughly a quarter of what the peer engine flags at my default threshold turns out to be benign collaboration or a shared textbook, which is what a review process is for. And the AI per-file scores drift across semesters as the models change, which means a threshold that worked in spring 2024 will not necessarily hold in fall 2026. Re-baseline once a year against a set of submissions you have already adjudicated. If your detector cannot reproduce its own past judgments, that is worth knowing before you use it on a student.
Codequiry is the one I have used for this the longest, in part because the peer, web, and AI scores land on the same record, so the calibration sheet is one export instead of three. The comparison with MOSS is worth reading if your department is deciding whether the cost is justified by anything other than convenience. For most courses the answer is the artifact: a report you can reopen, defend, and hand to a colleague.
What the scores cannot tell you
They cannot tell you whether a student used a model to explain a concept, which is study help and often good. They cannot tell you whether the fifteen minutes the student spent asking a model to debug their own recursion counts as assistance under your policy. That line is yours to draw, in writing, before the assignment goes out.
What the scores can do is make the conversation specific. "Two of your nine files score above 0.9 and I would like you to walk me through the bounds check in your merge" is a conversation a student can have with you. "We think you cheated" is not.
Frequently Asked Questions
Can I run one scan and check everything at once?
You can, and most platforms will let you request peer, web, and AI engines in a single job. The reason to think of them as separate pipelines is diagnostic: when a case comes back flagged, you need to know which engine produced the signal, at what threshold, and against what corpus. A single blended score hides all three.
How much does renaming variables hide from a peer scanner?
Very little, if the tool compares fingerprints or abstract syntax trees rather than raw text. Renaming identifiers changes token values but not the shape of the loops, the control flow, or the sequence of operations. It hides a lot from a naive text diff and almost nothing from MOSS, JPlag, or Codequiry's structural comparison.
Should AI detection scores be used as the only basis for an integrity finding?
No. Treat a high per-file score as a reason to have a conversation and to look at the code yourself, the same way you would treat a 70 percent peer match on a single helper. The score ranks your reading time. It does not replace it.
How do I handle a submission that scores high on AI for only one file?
Ask about that file specifically. Partial generation is the most common pattern I see: the student writes the structure and asks a model for one tricky function. That is exactly the case a per-file workflow surfaces and a submission-level average buries.
If you want to see the whole workflow on your own submissions before committing a semester to it, run a check on a past assignment and compare what it returns against the cases you already adjudicated by hand.