Last spring I was pulled into a contractor code verification project at a 70-person logistics software company in Columbus. I'll call the company Midway Fleet Systems; the CTO asked me not to use the real name because the contractor is still under NDA. The ask was simple. Before they paid a $26,400 invoice for a 14,000-line Python dispatch service, someone had to confirm the code was actually the contractor's work. Not a legal case. Just due diligence.
The senior engineer who flagged it had noticed small things. Comments in two languages. Some functions had no tests but were written like a library maintainer had spent a weekend on them. Others had tests but looked like a first-year bootcamp project I would have graded. That inconsistency is normal across a large pull request from a team, except this was a single contractor's final delivery. The CTO wanted a second pair of eyes.
What a Manual Contractor Code Review Actually Finds
I spent the first afternoon doing what any open source maintainer would do. Grep for unique string literals. Skim the imports. Check the commit history. The repo had one giant initial commit, so history was useless. The string literals turned up nothing on GitHub or Stack Overflow. The structure looked cohesive enough, but it also looked like code that had passed through someone else's hands before. That feeling is hard to act on without evidence.
A code plagiarism checker that only compares line-level text would have missed most of what we eventually found. The contractor had renamed every variable, changed function order, swapped loops for comprehensions, and reflowed comments. Superficial diff tools see almost nothing. But the bones of the code, the decisions about how to break a problem into functions and how those functions call each other, were still there.
How Token-Based Fingerprinting Works in a Real Review
Token-based fingerprinting normalizes source code into a sequence of symbols that ignores the easy-to-change surface details. Variable names become ID. String literals become STR. Numeric constants become NUM. The remaining tokens are structural: function definitions, control flow keywords, call expressions, assignments. Then you hash overlapping windows of those tokens, usually called k-grams. If two files share a high percentage of the same k-gram hashes, they likely came from the same original.
My throwaway implementation looked like this, and I want to be clear it is not production-grade, but it explains the idea well enough:
import ast, hashlib
def normalize_tokens(source):
tree = ast.parse(source)
tokens = []
for node in ast.walk(tree):
if isinstance(node, ast.Name):
tokens.append("ID")
elif isinstance(node, ast.Str):
tokens.append("STR")
else:
tokens.append(type(node).__name__)
return tokens
def kgram_hashes(tokens, k=5):
return [
hashlib.md5(":".join(tokens[i:i+k]).encode()).hexdigest()
for i in range(len(tokens) - k + 1)
]That script strips identifiers and string values but keeps the shape of the abstract syntax tree. Token normalization is not a search for identical text. It's a search for identical decisions after you strip the names people changed. The k-gram hashing part is what makes it fast enough to compare thousands of files against thousands of other files without an expensive diff.
I mentioned on our first API call I forgot to set the language parameter to Python, and the tokenizer treated the whole repo as JavaScript. Scores came back in the forties for everything. Once I set language=python, the real matches showed up. That wasted an afternoon I would rather have spent reviewing the first batch, but it also forced us to look at the settings instead of trusting defaults.
Contractor Code Verification Caught a 61% Match Hidden by Refactoring
The first meaningful hit was against a public GitHub repository for a dispatch routing library. The contractor had changed the package name, split several large modules into smaller files, renamed almost every identifier, and converted synchronous calls to async where the library allowed it. But the call graph and the internal decision points matched at 61% across the project. That number alone does not prove anything. The side-by-side view did.
When we opened the comparison, the pattern was unmistakable. A function originally named compute_distance_matrix had become build_route_grid, but it took the same six arguments in the same order, performed the same two-phase filtering, and returned the same nested dict. The comments had been rewritten, but the structural whitespace around the second loop was identical. That last detail matters. People can change names easily. They rarely change the shape of their code by accident.
We ran the same check against the open web through Codequiry's detect code plagiarism workflow. That surfaced two additional functions copied from a tutorial on real-time vehicle tracking. Those were small, about 120 lines total, but they carried the tutorial author's unusual variable names. The contractor had left one comment from the original tutorial in Spanish. That was the detail that initially made the senior engineer suspicious.
None of this was the kind of thing that makes headlines. No lawsuit, no bankruptcy. The CTO withheld the final milestone payment, asked the contractor to replace the copied portions, and renegotiated the delivery timeline. The code that remained original was actually good. The problem was the unoriginal parts had been mixed in so thoroughly that a normal review would have missed them.
Where the Thresholds Get Tricky
We started by reviewing every file pair above 85% similarity and every web match above 60%. That turned out to be too aggressive. Boilerplate inflates similarity. A 40-line file that is mostly imports and config can hit 90% against another file for reasons that have nothing to do with plagiarism. We eventually lowered the file-level review threshold to 70% and asked for a minimum of 25 substantive lines before we would look at a match. That cut the false positive review queue in half.
I have not tested this threshold on a corpus larger than a few hundred files, so treat it as specific to this project. The important point is not the exact number. It is that you need a review stage. A similarity score is a filter, not a verdict.
The score breakdown in Codequiry gave us a more useful view than a single percentage. We could see how much of the match came from peer similarity versus web similarity, and which files drove the overall score. That helped us explain findings to the CTO in plain language. A file with 94% web similarity to a known GitHub repo is a different conversation than a file with 72% similarity to another internal file that may have been a shared starter template.
What We Learned About Accepting Outside Code
The team changed the contract language for the next contractor. They now ask for a code provenance statement before each milestone payment, and they run a source code plagiarism checker as part of the merge review, not after the invoice arrives. The engineering director put it this way in the postmortem:
We were not looking for someone to fire. We were looking for a way to verify that the code we were about to put into production was actually ours to own. The tool just made the conversation easier.
That distinction matters. Contractor code verification is not about catching a villain. It is about knowing what you are licensing, maintaining, and responsible for. If a contractor pastes a function from a GPL-licensed repo and you ship it inside a proprietary service, the legal risk is yours. If they copy from a client they previously worked for, the trade secret risk is yours. The earlier you catch that, the cheaper the fix.
From my side as someone who reviews pull requests for open source projects, the process felt familiar. We already normalize code in our heads when we review. We ignore variable names and formatting and focus on behavior. Token-based fingerprinting just does that mechanically, at a scale a human cannot. The main lesson I took away is that refactoring hides plagiarism from text matching, but not from structure matching.
The second lesson is that a tool report is not the end of the review. It is the start. We spent more time investigating the matches than we spent running the scan. That is how it should be. If your workflow treats a similarity score as a final answer, you will either miss real issues or chase noise.
If you are on the hook for verifying contractor or vendor code, Codequiry's code plagiarism checker is a solid place to start. It does the token and AST normalization we needed, checks against web sources and peer submissions, and gives you a review queue that respects the fact that a human still has to make the call.