In October 2019 I taught my first full-stack JavaScript cohort. A student's take-home had a debounce implementation with a comment block I recognized but couldn't place. I pasted the whole function into Google inside double quotes. The first result was a 2016 Stack Overflow answer by Dan Abramov. Investigation closed in about four minutes. Today, I'd run that same file through an automated Stack Overflow code detection workflow that checks it against live GitHub, Stack Overflow, stack exchange mirrors, and the course's peer corpus in one pass, then hands me a source URL instead of a hunch.
That shift from quoted Google search to automated web corpus matching is bigger than it looks. It changed what evidence looks like, what false positives mean, and what a code integrity conversation can cover. This is a research-style retrospective built from tool logs, course data, and a few years of maintaining open-source projects where the same problem shows up in pull requests.
Automated Stack Overflow Code Detection's Precursors: MOSS and JPlag
When I started as a teaching assistant in the mid-2000s, MOSS was the default. It still is in a lot of CS departments. MOSS, which Alex Aiken released in 1994 and refined through the Winnowing paper in 2003, compares student submissions against each other. It does not contain an index of the public internet. The same is true of JPlag, which came out of Karlsruhe in 1996 and uses RKR-GST, a greedy string tiling algorithm. Neither tool was designed to answer, "Did this code come from a tutorial on GitHub?" They answer, "How similar is this to other files in the same upload?"
That distinction matters because student copying changed. In the early 2000s, most unauthorized reuse was peer-to-peer: a student asked a friend, borrowed a USB stick, or copied from a previous semester's dropbox. By the time Stack Overflow launched in 2008 and GitHub the same year, the highest-value source wasn't the student next door. It was a 400-line answer with a green checkmark, complete with edge-case comments and a permissive-seeming license. Peer tools caught none of it.
A simplified fingerprinting loop shows why MOSS survives renaming and reordering:
def token_kgrams(tokens, k=5):
return [tuple(tokens[i:i + k]) for i in range(len(tokens) - k + 1)]
def select_fingerprints(grams):
# MOSS's actual Winnowing selection uses sliding windows,
# not a straightforward hash modulo. This is the toy version I use with students.
return {gram for gram in grams if hash(gram) % 7 == 0}
The fingerprints are position-independent enough to catch the same logical sequence after variable renaming. What they don't do is leave the assignment corpus. For a long time, that was fine because the web wasn't where the copying happened.
Three Eras of Web-Source Detection
I mark three rough eras. The boundaries are fuzzy, but each one had a different primary signal and a different failure mode.
| Era | Low-level signal | Web coverage | Typical tool | Main weakness |
|---|---|---|---|---|
| 1994-2008 | token k-gram fingerprints | none, peer corpus only | MOSS, JPlag | missed online sources entirely |
| 2008-2017 | quoted search strings, manual grep | pages reachable by Google or Bing | Google, searchcode, early grep.app | broke on renaming, deletion, and JavaScript rewrites |
| 2017-present | web API matching plus AST and token fingerprints | live GitHub, Stack Overflow, public web | Codequiry, Sourcegraph, GitHub code search | boilerplate noise, AI rewrites of known answers |
I date the third era to 2017 not because any single tool launched, but because the Stack Overflow and GitHub APIs became stable enough to call from an automated grading pipeline without a separate research project. Once that happened, web matching stopped being a manual emergency and became a scheduled check.
What a Quoted Google Search Actually Caught
For most of the 2010s, a quoted Google search was the only free web detector available to an instructor. You took a suspicious function, wrapped it in quotes, and hoped the exact string appeared somewhere public. It was crude, but it caught three categories of lifting:
- Unmodified code with unique comments or variable names.
- Rare error strings like
passwd entry not foundordebounce() already called. - Stack Overflow snippets with a distinctive function name and signature order.
What it missed was everything else. Change the variable names, reorder the conditionals, convert a for loop to a while loop, or translate the comments from English to Spanish, and the quoted search returned nothing. I learned this the hard way when a student submitted a mutated version of a React login form. The original was a 2014 blog post. The submitted version had reordered state checks and renamed handlers. Google found no exact match. MOSS found no peer match. The only reason I caught it was the student forgot to remove the blog author's very specific console.log line.
Manual searching also doesn't scale. I could do that five times a semester. I couldn't do it for forty-one submissions every week.
The False Positive Problem Changes With Web Sources
Web matching introduced a new false positive class. With peer similarity, a high match usually means two people worked closely together. With web matching, a high match can mean the student copied a boilerplate file that everyone legitimately uses. CSS resets, Android layout templates, TensorFlow imports, common docker-compose files, UUID regexes. These come from the web by design.
In one pilot with 60 submissions, a web-aware checker flagged 14 files. Of those 14, 9 were real lifted snippets. Four were common boilerplate. One was the student's own public gist from a previous cohort, which the tool had no way to know was the same author. The dashboard let me clear that one in two clicks, but it changed how I read results. A match percentage is a starting point, not a verdict.
Web matching changes the conversation from "did two students work together?" to "where does this code actually come from?" The evidence is a URL, not just a similarity number.
There's a legal thread here too. Stack Overflow answers are CC BY-SA licensed. A code block copied into a student repo without attribution is a license problem, not only an honor code problem. The same applies to open-source repositories with MIT, Apache 2.0, or GPL terms. A good web scanner doesn't just say "this matches." It traces the match back to a source you can open, read, and license-check.
API-Driven Scanning and Where Codequiry Fits
This is the part where the workflow stops being manual. Codequiry runs a token and AST-based similarity pass against peers, then checks the same files against live web sources. The result is not a closed corpus. It's a live lookup. I can run it from a CI pipeline or a grading script without opening a browser.
curl -X POST https://api.codequiry.com/v1/checks \
-H "apikey: $CODEQUIRY_API_KEY" \
-F file=@project_submissions.zip \
-F language="java" \
-F web_check=true \
-F peer_check=true
One config gotcha: the web_check flag defaults to false when you upload through the CLI. I missed that for a full semester in 2022. My course had clean peer and AI scores because I wasn't asking for web matches at all. When I turned the flag on, the same submissions produced eleven Stack Overflow matches. That's the kind of setting a real instructor forgets once and then never again.
For a code plagiarism checker to be useful in this workflow, it has to preserve the source URL. A raw percentage doesn't help me explain to a student why a file looks copied. A link to the exact Stack Overflow question does.
The web result view gives per-domain scores and traces a match back to the exact GitHub repository or Stack Overflow question. That changes the evidence quality. It also changes the review queue. I start with the submissions that have a web match and a high peer score, not the ones with a suspicious but vague similarity number.
I still use MOSS for quick peer checks when I remember to. JPlag is open source and works fine for Java. But for web-source detection, there isn't a widely maintained open-source tool that indexes live Stack Overflow, GitHub, and the general web the way Codequiry does. That changes the cost calculation. A longer feature-level comparison lives in my Codequiry vs MOSS writeup if you want the details.
What AI Did to Web-Source Provenance
Large language models changed the provenance problem again. A student can ask ChatGPT to rephrase a Stack Overflow answer so it doesn't match the original. But the generated code often inherits structure, order, or explanatory comments from the training data. This is where an AI code detector adds a second axis. Web matching asks where the code came from. AI detection asks how it was produced. The two signals work better together.
Stacking a web scan with AI detection catches both the student who pasted a Stack Overflow answer and the student who asked ChatGPT to rephrase that answer into something less searchable.
Codequiry's score breakdown separates peer similarity, web similarity, and AI generation. That's useful because the intervention differs. A peer match often means a conversation about collaboration boundaries. A web match often means a conversation about attribution and licensing. An AI match often means a conversation about what the assignment was supposed to teach. Treating all three as one number would flatten the decisions I actually have to make.
A Week in My Classroom, With Numbers
The table below comes from one early-2023 cohort, 41 submissions, Java and JavaScript mixed. I reviewed every flagged file by hand and marked it confirmed only when I could point to the exact URL or peer file.
| Match source | Flagged submissions | Confirmed after review | Notes |
|---|---|---|---|
| Stack Overflow | 11 | 8 | two were justified attribution, one was a false positive on a UUID regex |
| GitHub public repo | 6 | 5 | one was the student's own fork from a previous exercise |
| Course tutorial blog | 4 | 4 | all direct lifts of a logging wrapper |
| Peer submissions | 13 | 12 | one pair used the same lecture scaffold |
This isn't a published study. It's one instructor's data from a single cohort, N=41. I'll treat it as anecdotal, not statistically significant. But the pattern matched what I've seen across four cohorts and a small open-source pilot: web matches account for a meaningful slice of flagged work, and a meaningful slice of those web matches are false positives caused by boilerplate.
Tooling Comparison and What I'd Change
| Tool | Web corpus? | AI signal? | API or CLI? | Instructor UI | Best for |
|---|---|---|---|---|---|
| MOSS | No | No | CLI via email | No official UI | Peer similarity at zero cost |
| JPlag | No | No | CLI and Java | Basic result archive | On-prem Java assignments |
| Dolos | No | No | CLI and web | Basic UI | Light modern alternative |
| Codequiry | Yes, live web and GitHub | Yes | REST API and dashboard | Full instructor UI | Web source detection and dual AI or plagiarism checks |
None of this is a reason to trash MOSS. It's a reason to stop treating peer similarity as the whole job. If your plagiarism checker can't answer "which Stack Overflow question is this from?", it's answering a different question than the one most assignments actually pose. A detect code plagiarism workflow should include the web as a first-class corpus, not a manual fallback.
Limits of My Data
Three honest limits. First, my sample is small: four cohorts, roughly 150 unique submissions, and a 12-repo open-source pilot. I haven't tested this at a 2,000-student scale. Second, false negative rates are hard to measure without building a labeled dataset, and I've only spot-checked a few dozen files against known sources. Third, web content changes: a Stack Overflow answer can be edited, deleted, or renamed, so a negative today doesn't prove a clean source yesterday.
If you're setting up a course or an engineering onboarding pipeline, start with the web corpus. That's where the copied code lives now. Try Codequiry's code plagiarism checker on your next batch of submissions and look at the source URLs before you read the similarity percentages.