I've spent the last three semesters wiring code plagiarism checker runs into our GitHub Classroom stack, and I want to show you the exact steps so you can copy the workflow in about thirty minutes. We run a Java-based CS200 course with 240 students, six TAs, and a per-assignment grading budget of roughly four hours across the team. Before the automation, a TA would manually pull every repo, zip it, upload to a similarity tool, wait, then dump the report. That was maybe ninety minutes per assignment. Now it's a scheduled GitHub Action that produces a CSV before we even open the grading queue.
The core idea is simple: on every student push, a workflow script calls Codequiry's API, sends the changed files plus the assignment context, and writes a per-submission similarity and AI score into a CSV artifact. You don't need a server. You don't need to leave GitHub. You do need an API key and a small Python script, and I'll give you both below.
What the workflow actually does
When a student pushes to a repository in your GitHub organization, the action runs a Python script that gathers the relevant source files (we use src/main/java for the assignment), packages them into a single submission, and sends them to Codequiry's /v2/check endpoint with a few parameters. The key parameters are peer_score=True to compare against the rest of the class, web_score=True to check public GitHub, Stack Overflow, and other web sources, and ai_score=True to run the AI code detector on each file. You also pass a course_id and assignment_id so the results land in the right bucket.
One flag I forgot for a full assignment cycle: include_web=True. Without it the API only returns peer matches, which meant we missed three students who had copied from a public tutorial repo. The web-source flag is easy to overlook if you copy the minimal example from the docs, so add it from the start.
The GitHub Actions YAML file
Here is the workflow file I drop into a new private repo called plagiarism-runner that we trigger on a schedule or manually per assignment. I keep it separate from student repos so students never see the scoring logic. The workflow is intentionally boring: checkout the runner repo, set up Python 3.11, install the requests library, run the script, and upload the CSV artifact.
name: Run Codequiry Plagiarism Check
on:
workflow_dispatch:
inputs:
assignment_id:
description: 'Assignment ID in Codequiry'
required: true
course_id:
description: 'Course ID in Codequiry'
required: true
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- run: pip install requests pandas
- name: Run similarity and AI scan
env:
CODEQUIRY_API_KEY: ${{ secrets.CODEQUIRY_API_KEY }}
ASSIGNMENT_ID: ${{ github.event.inputs.assignment_id }}
COURSE_ID: ${{ github.event.inputs.course_id }}
run: python scanner.py
- uses: actions/upload-artifact@v4
with:
name: codequiry-results
path: results.csv
The manual trigger with workflow_dispatch means you don't need a cron job firing every hour when nobody is submitting. We run it once after the deadline, or twice if we want an early look at the weekend. The environment variables keep the API key out of the YAML, which is the only part I insist on from a security perspective.
The Python scanner that calls Codequiry
This is the script, trimmed to the parts that matter. It iterates over student repos via the GitHub API using a personal access token, collects Java files, sends them to Codequiry, and waits for the scan to finish before writing results. I've stripped error handling and retries to keep it readable, but in production I add a 60-second timeout and a retry on 429 rate limits.
import os, time, csv, requests
from pathlib import Path
GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
CODEQUIRY_KEY = os.environ["CODEQUIRY_API_KEY"]
ASSIGNMENT_ID = os.environ["ASSIGNMENT_ID"]
COURSE_ID = os.environ["COURSE_ID"]
headers = {"Authorization": f"Bearer {CODEQUIRY_KEY}"}
base = "https://api.codequiry.com/v2"
# Get list of student repos from the GitHub org (simplified)
org = "cs200-fall2025"
repos = requests.get(f"https://api.github.com/orgs/{org}/repos?per_page=100",
headers={"Authorization": f"token {GITHUB_TOKEN}"}).json()
results = []
for repo in repos:
if not repo["name"].startswith("assignment4-"):
continue
# Collect Java files from the default branch
files = []
# ... clone or fetch each repo, walk src/main/java ...
# For brevity, assume we have a local path variable
submission_payload = {
"course_id": COURSE_ID,
"assignment_id": ASSIGNMENT_ID,
"files": files,
"peer_score": True,
"web_score": True,
"include_web": True,
"ai_score": True,
"language": "java"
}
resp = requests.post(f"{base}/check", json=submission_payload, headers=headers, timeout=60)
check_id = resp.json()["check_id"]
# Poll until done
status = "running"
while status == "running":
time.sleep(5)
status_resp = requests.get(f"{base}/check/{check_id}", headers=headers)
status = status_resp.json()["status"]
report = requests.get(f"{base}/check/{check_id}/report", headers=headers).json()
results.append({
"repo": repo["name"],
"peer_score": report.get("peer_score", 0.0),
"web_score": report.get("web_score", 0.0),
"ai_score": report.get("ai_score", 0.0),
"top_peer_match_repo": report.get("top_peer_match_repo", ""),
"top_web_source": report.get("top_web_source", ""),
})
with open("results.csv", "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=results[0].keys())
writer.writeheader()
writer.writerows(results)
The polling loop is the only mildly annoying part. Codequiry's scan on 200 Java files takes about 40 seconds for peer plus web, and an extra 15 seconds when AI detection is enabled. We poll every five seconds, which means the whole class finishes in under a minute per repo even with network overhead. The CSV output is deliberately flat: one row per student, four columns of scores, plus the top match source. That's enough for the triage spreadsheet.
Why AST matching catches what text matching misses
Here is the thing I explain to new TAs every semester: code plagiarism is not about identical characters; it's about identical structure. A student who renames every variable, reorders methods, and changes whitespace will fool a text-based diff but not a token plus AST comparison. Codequiry's engine converts each Java file into an abstract syntax tree, discards identifier names and formatting, and fingerprints the control flow. Two submissions that differ only in superficial naming produce a very high structural similarity score.
Here is a concrete pair from our fall 2025 assignment 4. The original on the left, the "refactored" version on the right:
// Original submission
public int computeTotal(List<Item> items) {
int sum = 0;
for (Item item : items) {
sum += item.getPrice() * item.getQuantity();
}
return sum;
}
// "Refactored" submission
public int calculateTotal(ArrayList<Element> elems) {
int accumulator = 0;
for (Element e : elems) {
accumulator = accumulator + e.price * e.qty;
}
return accumulator;
}
Text similarity between those is almost nothing. Token plus AST similarity is 91% because the loop structure, the accumulation pattern, and the multiply-add inside the loop are identical. When we saw a pair like this, the dashboard flagged it immediately. The old MOSS output would have shown something like 22%, and a busy TA would have missed it.
The misconception worth killing here: refactoring to hide plagiarism is a known strategy, and most tools that only do line-level comparison let it through. If your current checker just runs diff or a string-similarity algorithm, you are catching only the laziest copiers. The students who put effort into hiding their copying are exactly the ones who should be detected, because they've demonstrated enough understanding to know what to change.
Adding AI detection to the same workflow
Starting in spring 2025 we added ai_score=True to the same API call. That runs Codequiry's AI-written code detector on each file, producing a probability that the code was generated by ChatGPT, Copilot, or Claude. The output is a per-file score and an aggregate for the submission. We use a threshold of 0.75 for a flag, not a verdict. Scores between 0.5 and 0.75 get a manual look if the peer similarity is also high, because that usually means a student generated code, then tried to clean it up, then copied a classmate's generated code.
One honest caveat: we have not tested this past a few hundred submissions per assignment, and the AI detector does produce occasional false positives on very formulaic Java boilerplate. But because we stack AI score with peer and web similarity, a high AI score alone is rarely our sole evidence. The stronger signal is AI score plus zero web matches plus a near-identical peer match, which usually means one student generated the same solution and shared it.
Triage in the grading spreadsheet
After the action finishes, the CSV artifact is downloaded, and I do a three-minute spreadsheet setup. Sort by peer_score descending, add conditional formatting on peer_score and ai_score above 0.7, and color the row red if either is high. That usually surfaces five to eight submissions out of 240 that need a closer look. Each close look takes me about forty seconds: open the repo, compare the flagged files side by side using the link in the report, and decide. Most are confirmed within two minutes. The rest of the submissions I never open individually; the numbers are low enough to trust.
I used to think this kind of automation was overkill for a single course. But with six TAs spread across different sections, the consistency matters. One TA might spot a suspicious pair, another might not. The CSV gives us a shared, sorted list, and everyone starts at the top.
If you already use MOSS or JPlag, the migration path is straightforward: keep the existing tool for historical comparison, add Codequiry for the peer plus web plus AI combination in one pass. The Codequiry vs MOSS page has a more detailed comparison, but the short version is that Codequiry gives you a real dashboard, a web-source check, and AI detection without any Perl scripts or server setup.
One configuration detail people forget
The single most common support question I get from other instructors is why their web matches are empty. Nine times out of ten it's the include_web flag missing from the payload. The API default is peer-only unless you explicitly opt in, which is a sensible default for privacy but catches people off guard. Add the flag, run a small test on one repo, and confirm you see a web score before running the whole class.
The second detail is git file exclusions. If your student repo includes boilerplate files or starter code, exclude them from the scan or you'll get inflated similarity from the starter package. I pass a list of file paths to ignore, usually build/ and src/test/. In the script above, that filtering happens before building the files list, which I skipped for brevity. The production version walks only src/main/java and skips anything under src/test.
After three semesters of running this workflow, the total time per assignment dropped from roughly ninety minutes to ten: five minutes to trigger the action and wait, three minutes to triage the flagged rows, two minutes to write an email to the academic integrity officer when needed. The system isn't perfect, and I still eyeball every high score, but it has caught cases we would have missed entirely, including one student who copied from a two-year-old GitHub repository that none of us had ever seen.
If you are teaching a programming course and still doing manual checks, start with the plagiarism checker for code product page to see the API docs and get a key. The setup is an afternoon, and the time savings compound every single assignment after that.