The assignment was a spell checker. Four hundred and twelve submissions from a two-course intro Python sequence, every one of them implementing Levenshtein distance, all due on the same Friday in October 2024. The routine peer scan came back with eleven clusters, which for a cohort that size is unremarkable. Then the coordinator ran a web-source scan and the count went to sixty-one.
To detect code copied from Stack Overflow or GitHub, you have to look somewhere a peer-to-peer tool never looks. MOSS, JPlag and Dolos compare submissions against each other. None of them indexes the open web, and none can tell you that nine students lifted the same dynamic-programming loop from a tutorial page. A web-source scan is a separate pass with separate infrastructure, separate rate limits, and a far worse false-positive rate, because the internet is a shared corpus and plenty of it is legitimate.
Nothing about that is obvious to a department buying its first tool. What follows is the workflow one intro sequence settled on after two semesters of getting it wrong.
Why a peer scan will not catch code copied from Stack Overflow
MOSS is the reference implementation and has been since 1994, when Schleimer, Wilkerson and Aiken published the winnowing algorithm behind it. The tool tokenizes source, hashes overlapping k-grams, selects a subset of those hashes with a sliding window, and reports documents whose fingerprints overlap more than chance would allow.
That design is why MOSS survives renaming and reformatting. It is also why MOSS has nothing to say about provenance. The Stanford server receives a batch, compares the files to each other, emails back a URL, and forgets the batch. JPlag 5.x, maintained at KIT, and Dolos, out of ETH Zurich, work the same way: a closed submission set, a similarity score, a pair of files.
Web checking is a search problem wearing a similarity problem's clothes. You need an index or an API, a normalization pass to strip formatting noise, and a ranking scheme that separates a forty-token match inside a dynamic-programming loop from a twelve-line match on print("Enter a string:").
"The peer scan told me eleven students had a problem," said Priya Raghunathan, who coordinates the intro sequence at a 22,000-student public university in the Midwest. "The web scan told me sixty-one did. Both numbers were wrong, and I had a week to work out which one was less wrong."
Step 1. Normalize the submission set before anything else
Raw LMS exports are inconsistent. Jupyter notebooks carry execution counts, cell metadata and timestamps that differ between two students who submitted identical code. Zip files arrive nested two deep. Compiled class files ride along with the Java sources. None of this changes a scanner's judgment, but all of it corrupts exact-duplicate detection and the evidence trail.
import hashlib, json, pathlib
root = pathlib.Path("submissions/fall2024/hw3")
seen, index = {}, []
for path in root.rglob("*.py"):
if ".ipynb_checkpoints" in path.parts or "__pycache__" in path.parts:
continue
raw = path.read_bytes()
digest = hashlib.sha256(raw).hexdigest()
student = path.relative_to(root).parts[0]
if digest in seen:
index.append({"student": student, "file": str(path),
"exact_duplicate_of": seen[digest]})
continue
seen[digest] = student
index.append({"student": student, "file": str(path), "bytes": len(raw)})
pathlib.Path("manifest.json").write_text(json.dumps(index, indent=2))The manifest gives you two things the scan cannot. First, a fast exact-duplicate prefilter that catches the laziest copying before any similarity analysis runs. Second, a stable identifier per submission, so a match discovered in week nine can be traced back in week fourteen when the grade appeal lands.
For Java and C++, strip compiled output and keep the original source. For notebooks, export to .py but retain the .ipynb alongside it. The cell metadata is sometimes the evidence.
Step 2. Run the peer comparison first and keep the output
Order matters more than people expect. A student who copies from a public GitHub repo also matches every other student who copied from the same repo. The peer pass reports a cluster of six and it looks like collusion. It isn't. It's six independent instances of one web source, and without the web pass you cannot tell the difference.
Run the peer pass, save the raw output, and do not open disciplinary conversations from it. Then run the web pass and reconcile. A cluster that dissolves into a single URL is a different problem from a cluster with no external source, and the two call for different conversations with different students.
Reconciliation is the part most departments do by hand in a spreadsheet, and it is where the hours go. Codequiry's code plagiarism checker scores peer similarity, web matches and AI generation in a single pass and attaches the source URL to each web hit, which removes the reconciliation step rather than automating it. That distinction matters at four courses and 1,600 submissions a term.
Step 3. Decide which web sources are in scope
Nobody scans "the web." You scan a set of indexed sources, and the set you choose determines what you will and will not find.
| Source | Why it matters | Signal quality |
|---|---|---|
| Stack Overflow | Highest-volume single source for intro assignments | High, and archived by the Wayback Machine |
| GitHub public repos | Prior-year uploads, course solution repos, student portfolios | High on distinctive code, noisy on boilerplate |
| GeeksforGeeks, Programiz, TutorialsPoint | Algorithm walkthroughs with runnable code blocks | High |
| Rosetta Code | Canonical implementations in 90-plus languages | Very high, docstrings included |
| Other universities' course sites | Lecture code and posted past solutions | High |
| Chegg, Course Hero | Paywalled and crawler-hostile | Low, mostly invisible |
Two channels are systematically invisible. Video tutorials put the code on screen and never in text. Discord servers and group chats are not crawlable at all. A clean web report is evidence of nothing where those channels are concerned.
Step 4. Configure API access to GitHub code search
GitHub's code search API rejects unauthenticated requests outright, and the authenticated limit for that endpoint is 10 requests per minute. Budget accordingly, and run the job overnight the first time.
curl -s -H "Accept: application/vnd.github.text-match+json" \
-H "Authorization: Bearer $GITHUB_TOKEN" \
"https://api.github.com/search/code?q=%22def+levenshtein%22+language:python&per_page=30"Build queries from distinctive tokens rather than from whole files. A search for def levenshtein returns tens of thousands of results, and GitHub caps any single query at 1,000. A search for an unusual helper name or a distinctive comment string narrows it to something a human can read. Three or four hand-picked queries per assignment beat an automated file-to-query pipeline, and they take twenty minutes to write.
Record the retrieval timestamp for every hit. Repositories get edited and deleted, and a match you cannot reproduce three months later is a match you cannot defend.
Step 5. Rank matches by matched tokens, not by percentage
Percentage similarity is the wrong sort key for web matches. Twelve lines of standard input boilerplate score 100%. A forty-token match inside a hand-rolled DP table might score 18% of the file.
for i in range(1, m + 1):
for j in range(1, n + 1):
if s1[i - 1] == s2[j - 1]:
dp[i][j] = dp[i - 1][j - 1]
else:
dp[i][j] = 1 + min(dp[i - 1][j], dp[i][j - 1], dp[i - 1][j - 1])Rename dp to table, i to row, reindent the block, and a text matcher's score collapses. A token fingerprint does not move, because the token sequence is unchanged. That is the argument for AST and fingerprint comparison over string similarity, and it is precisely the case where a generic checker returns a clean bill of health on copied code.
Sort the review queue by longest contiguous matched token run and work from the top. The cutoff that worked here was 25 tokens; anything shorter went to the bottom and usually stayed there. That number came from a single semester of tuning, and no one has published a threshold that transfers across assignments.
Step 6. Triage in a diff view and build an evidence packet
The token count gets you to the right file. The diff gets you to the right conclusion.
Raghunathan's review checklist has four items on it, and none of them is the similarity score. The same bug: two submissions that both fail on the empty string, in the same way, on the same line, are not independent work. The same comment typo. The same unusual optimization, such as an early-exit band in the Levenshtein table that no intro student invents. The same unused import, left behind by whoever wrote the original.
Every flagged submission gets an evidence packet: source URL, retrieval timestamp, a Wayback snapshot if one exists, matched token count, a side-by-side diff, and the submission identifier. In spring 2024 a case collapsed because the tutorial page had been edited mid-semester and the live URL no longer matched what the student had pasted. The archive was the only thing that held.
Step 7. Build an exclusion list before the semester starts
Starter files, the instructor's reference solution, textbook listings, license headers and language idiom all generate matches that mean nothing. The exclusion list handles them, and it has to exist before the first scan, not after the first appeal.
The entry everyone forgets is the one for files the course itself distributed. Five students in the fall 2024 run were flagged for code the instructor had posted as a worked example on the course site. That false positive costs an apology and a fair amount of goodwill with a cohort that compares notes.
There is a licensing dimension too. Stack Overflow content posted after 2018 is CC BY-SA 4.0; earlier content is 3.0. A GitHub repository with no license file is all rights reserved by default, whatever the README implies. A student who paste-copies a function without attribution has, in the strict sense, created a licensing problem for anything that code later ships inside. Most honor codes do not mention this. A handful of computer science departments have started to.
Step 8. Put the scan on a schedule
End-of-term batch runs are how this work gets abandoned. Someone exports 1,600 submissions in December, the scan takes six hours, and by the time the report is readable the grades are filed.
A weekly run against the current assignment changes the shape of the job. Matches arrive while the material is fresh, students get asked about them in office hours instead of at a formal hearing, and the instructor reviews twenty submissions rather than four hundred. Both Codequiry and MOSS can be driven from a script. Codequiry exposes a REST API and a CLI that fit a scheduled job, with the web dashboard handling review afterward; teams running this without engineering support tend to prefer the dashboard and a code plagiarism checker for teachers flow that never opens a shell.
Where the workflow stops working
Cross-language translation is the sharpest edge. A student who reads a Python implementation and rewrites it in Java produces a submission with no token overlap and no web match. Cross-language comparison exists in JPlag for a few language pairs and in research prototypes, but the noise level in an intro course is high enough that most departments leave the feature off.
AI-generated code is the other edge. A submission written by Claude or GPT-4 in 2025 is by definition not on the web, so the web pass returns nothing and the file looks clean. Pairing it with an AI code detector closes that gap, and the two signals are complementary rather than redundant: web matches establish provenance, AI scores establish that a human probably did not write the file. Neither is a verdict, and any vendor promising otherwise is overselling the mathematics.
What the 412 submissions actually cost
Sixty-one flagged. Thirty-eight substantiated: twenty-two matched a GeeksforGeeks article on edit distance, nine matched a Rosetta Code entry, four matched a prior-year repository at a different university, three matched a peer institution's course site. Twenty-three were noise. Eleven of those were license headers and textbook boilerplate, seven came from the exclusion list failure above, and five were fragments too short to matter.
Review time for the entire cohort, once the queue was sorted by matched tokens, ran a little over forty minutes. The previous year's spreadsheet exercise took six hours and missed more. No appeals were upheld, which is the number that matters to an integrity office, and that outcome traces to the evidence packets rather than to any score.
"The honor code says unauthorized assistance, which is a sentence a committee has to interpret," said the department's academic integrity officer, who asked not to be named while an unrelated case was open. "A scan gives us a document to argue about. It doesn't give us the argument."
Which is the right frame for the whole exercise. Web-source detection is triage. It splits 412 submissions into a short list worth reading closely and a long list that is probably fine, and it does that in under an hour instead of a week. Everything after the sort is human judgment, and the tool's only real job is to make that judgment defensible when someone asks how you knew.
Codequiry's code similarity and AI detection platform runs the peer, web and AI passes in one scan, and the guided demo ships with five sample Java submissions if you want to see the output format before uploading a real cohort.