When a 400-line Java service class arrived from our payment contractor on a Tuesday, it passed unit tests, Semgrep 1.66.0, and Trivy 0.50.1. The variable naming looked like a 2019 Hibernate tutorial. I didn't schedule a review meeting. I ran contractor code plagiarism detection in the merge pipeline, where Codequiry's API sits next to our other security checks and blocks merge on evidence, not suspicion.
A code review can tell you what a function does. A web provenance scan can tell you where it came from. In an outsourced engagement, the second one matters more.
Contractor code plagiarism detection in a merge gate
Contractors are not students, and the risk profile is different. A student who copies code gets an academic review. A vendor who copies a GPL-licensed GitHub snippet into our settlement service creates a legal cleanup that can cost more than the engagement. That is why I moved the plagiarism scan from a quarterly audit into the same GitHub Actions workflow that runs Semgrep and Trivy. We use Codequiry as a managed plagiarism checker for code because it checks peer similarity against our baseline, public web sources, and a separate LLM-generated code signal. A self-hosted JPlag server handles peer similarity fine, but it does not tell me that a Kotlin data mapper came from a 2019 Stack Overflow answer.
The old process was a spreadsheet, a manual JPlag run, and a lot of internal links. We caught things after offboarding, when remediation was already expensive. In January 2025 we wired the scan into the pull_request event for all vendor forks. Since then, the check has flagged nine PRs. Four were direct provenance matches on GitHub. Two carried high Copilot signatures. One was a false positive on a generated protobuf. The other two were inconclusive and got human review. That is a manageable queue.
A baseline snapshot keeps the web scan useful
I do not feed the entire monorepo into the scanner. I pass the diff set from the vendor branch, because our internal core Java libraries are full of patterns a generic web scanner would flag as similar to ancient tutorials. The Codequiry check gets the changed files only, and we exclude generated Gradle wrappers, XML schemas, and lockfiles. That keeps the report focused on human-written logic.
We also snapshot the current internal approved codebase once a week and store it as the peer reference. That way, when a contractor submits a file that matches our own payment engine, it is not marked as copied from the web. It is marked as an internal peer match, which we review separately. The distinction matters. A contractor copying our own code is usually a good sign if they were supposed to integrate with it. A contractor copying someone else's code is a different problem entirely.
The GitHub Actions workflow I actually run
Here is the workflow we run on every PR that touches vendor-contracted paths.
name: contractor-originality
on:
pull_request:
types: [opened, synchronize]
paths:
- 'vendor/**'
jobs:
codequiry:
runs-on: ubuntu-24.04
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- name: Run Codequiry plagiarism check
id: scan
uses: codequiry/[email protected]
with:
api_key: ${{ secrets.CODEQUIRY_API_KEY }}
mode: peer-plus-web
min_similarity: 65
fail_on: "peer_match"
- name: Attach evidence report
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: codequiry-report
path: codequiry-report.json
The fail_on key is the one everyone forgets. Without it, the action writes a report and exits zero, which means a direct GitHub match can merge silently. I learned that the hard way when v2.0.0 had a zero-byte report bug on branches with a large vendor tarball. We pinned to v2.1.0, set fail_on explicitly, and now the check blocks merge unless a human with write access overrides it.
I keep the API key in an environment secret. The action can run against our internal baseline and web sources without exposing the key to a forked branch. We also set the job to run on the merge queue, not on every push to a draft PR. That cut our scan minutes roughly in half because we are not rescanning files that have not changed.
Reading the evidence without opening every file
When the check fails, I do not want a similarity percentage. I want a synced diff that shows the exact lines and where they came from. Codequiry's evidence view does this. For one March 2025 PR, it showed an 87% match between a Java reconciliation service and a GitHub Gist from 2019, with variable names changed but identical control flow and comments. That was enough to reject the PR and ask the vendor for a provenance statement.
The report also contains per-source counts for web matches. We had a PR where the scanner flagged an 81% match to a Stack Overflow answer. The contractor claimed they wrote the code from memory, but the same comment typos were present. The evidence link made the conversation short. We did not argue about intent. We pointed at the source and asked for a license or a rewrite.
AI-generated code needs a separate threshold
Codequiry's AI code detector runs as a second scan in the same workflow. I do not treat a high AI score the same way I treat a peer match. An AI score alone does not prove the contractor pasted from Copilot. It means the statistical shape of the code matches what LLMs tend to produce. That is a review signal, not a legal claim.
We use a threshold of 60 for the per-file AI score to trigger a human review. Scores between 30 and 60 are visible but do not block. Scores above 60 on a substantial file require the vendor to explain the development process. We have not yet failed a whole engagement on AI score alone, but we have sent two files back for rewrite because the Copilot signature was strong and the logic was boilerplate CRUD that should not exist in our settlement layer.
The table view sorts submissions by AI score range, so we start with the highest outlier. That matches how I review security scans too. I want the risk distribution first, not a chronological list.
What I let through and why
I do not block peer matches below 70% unless the matched lines are meaningful. A 68% match on a generated Gradle file is noise. A 65% match on a 40-line Kotlin payment status mapper is not. The threshold is a starting point, and the evidence review decides. Every scanner produces false positives. Codequiry's have been rare enough that I can manage them manually.
We also maintain an exception file for vendor branches. If a contractor is extending a previously approved microservice, we whitelist that directory for the peer scan but not the web scan. That one rule cut the review queue in half. It also means the check does not become a rubber stamp. The web and AI scans still run, and they still block if something weird comes in.
Where this breaks down after a few hundred PRs
We have run this gate on roughly 300 vendor PRs since January 2025. I do not trust the numbers past that because the dataset is still small. The main gap is semantic equivalence. If a contractor reads a Stack Overflow answer and carefully reimplements the algorithm with different variable names, different control flow, and no shared tokens, no token or AST fingerprint will catch it. That is not a Codequiry limitation. That is a limitation of provenance detection in general.
The AI detector has a similar soft spot on heavily templated code. A Spring controller with standard annotations and three service calls will sometimes score high simply because there are only so many ways to write it. I treat that as an expected false positive pattern, and I do not send those to vendors as accusations. I add a note and move on. If the same vendor shows high AI scores across twenty files, that is when I ask questions.
If you are still running quarterly code provenance audits with MOSS or JPlag and a spreadsheet, it may be time to move to a code plagiarism checker that speaks HTTP and fails a build. Start by wiring Codequiry's API into one vendor branch and seeing what the report actually shows.