How a CS Professor Spots Refactored Code Plagiarism in Java Labs

At 11:40 p.m. on a Tuesday in October 2023, I had two CS 142 submissions open side by side. The first used a variable named runningTotal, a for loop that counted upward, and a helper method called isMultiple. The second used a variable named s, a while loop that counted downward, and no helper method at all. A surface comparison would have called them unrelated. They were almost certainly the same program. The tell was a redundant if check around the final println, a block that did nothing in both files. That is refactored code plagiarism: the kind designed to survive a first glance and a text-based comparison.

The short answer to how I catch it: refactored code plagiarism detection is not about matching characters. It is about comparing token sequences, control-flow structure, and assignment-specific fingerprints that persist after renaming variables, reordering methods, and converting loops. My weekly workflow has seven steps: collect, tokenize, run a structural comparison, sort by peer similarity, read the top diffs, check the web, and interview the student. I will walk through each step exactly as I run it in my Java course, including the scripts, thresholds, and the parts I still do by hand.

Step 1. Collect every submission and normalize filenames before anything else

I start with the LMS export. At my university, Canvas gives me a zip file that preserves file names, but students name their files things like Assignment3_final_FINAL(2).java. I do not trust filenames. I require every submission to be named netid_Assignment3.java before upload, and I have the same requirement for hand-graded labs. This takes five minutes on the first lab and saves an hour later.

mkdir -p submissions
for f in *.java; do
  netid=$(echo "$f" | cut -d_ -f1)
  mkdir -p "submissions/$netid"
  cp "$f" "submissions/$netid/Assignment3.java"
done
sha256sum submissions/*/Assignment3.java > manifest.sha256

I check the SHA-256 manifest first. Exact copies are rare in a course where students know I run a checker, but they do happen when two students shared a completed file without editing it. Those never need a similarity report. They go straight into my evidence folder.

One semester I learned this the hard way. A previous version of the MOSS client I used in 2019 dropped any file whose path contained a space, and six submissions silently disappeared from the run. Since then I rename everything to the netid pattern and never let spaces into the upload. I do the same inside Codequiry; the uploader accepts a zip of netid folders, and the folder names become the submission identifiers.

Step 2. Tokenize all files before comparing anything

A token is the smallest meaningful unit in a source file: keywords like public and int, identifiers like runningTotal, literals like 100, and operators like <= and %. Token-based comparison works by converting source code into these units, then comparing the sequences. The reason this catches refactoring is that renaming a variable changes one token, while the surrounding operator and keyword structure stays intact.

I run a small Python tokenizer over every file before the plagiarism check because it gives me a quick local baseline. The script is simple enough to audit by eye, and I keep it in the course repository so the TAs can rerun it if I am away.

import re
from pathlib import Path

TOKEN_PATTERN = re.compile(
    r'[A-Za-z_][A-Za-z0-9_]*'   # identifiers and keywords
    r'|\d+\.?\d*'                # numeric literals
    r'|"[^"]*"'                  # string literals
    r'|[{}()[\].,;:+\-*/%<>=!&|^~?]+'  # punctuation and operators
)

def tokenize(path):
    text = Path(path).read_text(encoding='utf-8', errors='ignore')
    return TOKEN_PATTERN.findall(text)

I then build 9-grams, consecutive runs of nine tokens, and compute a Jaccard similarity between each pair of submissions. The 9-gram window is long enough to reduce coincidental matches in short Java labs but short enough to catch copied segments of about 20 lines. With n=5, I see too many false matches. With n=15, the comparison becomes too strict and misses partially copied methods.

def ngrams(tokens, n=9):
    return {tuple(tokens[i:i+n]) for i in range(len(tokens) - n + 1)}

def jaccard(a, b):
    return len(a & b) / max(1, len(a | b))

For my 60-line assignment, a Jaccard overlap above 0.40 between two submissions is enough to make me suspicious. I do not use this alone. Token similarity catches renamed variables and some statement reordering, but it underweights larger structural changes like loop inversion or condition rewrites. That is exactly why the next step exists.

Step 3. Run the refactored code plagiarism check

The local script gives me a quick triage list, but it is not the final word. I upload the normalized zip to Codequiry and create a check called CS142_F23_Assignment3. I set the language to Java and leave the engine selection on peer, web, and AI because I want all three signals. Codequiry is a code plagiarism checker that runs token similarity, AST comparison, and fingerprinting against peer submissions and public web sources in one pass. This matters because my local token script cannot compare submitted files against GitHub, and the AST layer catches the structural changes my 9-gram script underweights.

I also upload the assignment's starter scaffold as a template file so Codequiry can subtract boilerplate before scoring. Without that step, the first half of the report is duplicates of my own starter code.

I still keep MOSS and JPlag in the tool drawer. MOSS 2020, the Stanford client, is fast and handles large classes well, but it normalizes identifiers and misses some method reordering unless I provide base files. JPlag 4.3.0 gives me excellent AST comparisons, but its interface is minimal and I have to host it myself. For a weekly lab, running three tools is not sustainable. Codequiry's combined report gives me the same layers in one place, and the output shows which checker produced each match. We compared the engines more concretely in our Codequiry vs MOSS notes.

Step 4. Sort the report by peer similarity and let the queue do the first pass

The report view ranks pairs by cohort outlier score, not by netid. I do not start at the top of the class roster. I start at the top of the peer similarity queue, where the system has already discarded pairs below my threshold. In a recent run with 94 submissions, the queue listed 23 pairs over 32%. Nineteen were obvious false positives from identical assignment scaffolding. Four needed a closer look.

The thresholds I use are specific to a 40-line assignment and a class of roughly 80 to 110 students. I have not validated them beyond my own course sizes, so treat them as starting points:

Peer similarity score	What I do
0 to 25%	Ignore
26 to 39%	Skim if the same names appear repeatedly across assignments
40 to 64%	Open the evidence review for the top pair
65% and above	Flag for investigation, then verify the diff by hand

I do not act on any percentage alone. The number ranks the queue. The diff makes the case.

Step 5. Read the diff and look for the pattern that should not be identical

Here is the original submission from the Tuesday night comparison. Note the accumulator, the ascending loop, and the straightforward divisibility check:

public class Assignment3 {
    public static void main(String[] args) {
        int runningTotal = 0;
        for (int i = 1; i <= 100; i++) {
            if (i % 3 == 0 || i % 5 == 0) {
                runningTotal += i;
            }
        }
        System.out.println(runningTotal);
    }
}

Here is the refactored version from the second submission. The variables are renamed, the loop is inverted, and the conditional uses De Morgan's law to test for non-multiples:

public class Euler1 {
    public static void main(String[] arguments) {
        int s = 0;
        int n = 100;
        while (n > 0) {
            if (n % 3 != 0 && n % 5 != 0) {
                n--;
                continue;
            }
            s += n;
            n--;
        }
        System.out.println(s);
    }
}

A character-based checker would report almost no similarity. The variable names are different, the loop construct changed, the terminal condition changed, and the divisibility check was rewritten. But the AST and token fingerprint reveal the same shape: an accumulator, a bounded iteration from 1 to 100, a conditional that admits multiples of 3 or 5, and a final print. When Codequiry opens the pair, the side-by-side diff marks exactly those corresponding subtrees.

Any two students who solved the assignment independently will share structure because the assignment is the same. The signal is in the parts that are not required by the prompt, and in the parts that survive refactoring for no reason.

Two students who both read the same Stack Overflow post will also show similarity, which is why the web check matters. But the strongest signal is a pair that shares a mistake or an odd structure that did not come from the assignment prompt. In the example above, the redundant if check around the final print is such a fingerprint. It does not change behavior. It survives refactoring only because the student copied it without understanding it.

For that pair, my local 9-gram Jaccard similarity came back at 0.34. Below my local triage threshold. Codequiry's AST score was 91%. That gap is the whole argument for using more than one layer.

Step 6. Check web and GitHub sources before forming a hypothesis

Similarity among peers can mean one copied from the other, or both copied from the same public answer. Before I email a student, I open the web matches tab and look for the highest domain, usually a GitHub gist or a Stack Overflow answer from a previous semester. In the same CS 142 run, one pair matched each other at 84% but matched a public GitHub repository at 88%. That changed my conversation completely: instead of accusing one student of copying another, I asked both why their submission was identical to a public solution.

A source code plagiarism checker that checks only peer submissions will miss this. You need web source matching for the full picture. The same tool run also flags whether a submission matches a published tutorial or an open-source repository, which matters when a student defends the code as original work.

Step 7. Talk to the student before writing the referral

I never send a plagiarism report to the honor board on the basis of a percentage alone. I schedule a 10-minute meeting during office hours, show the student the paired diff and the web match, and ask them to explain the code. In about half of the clear cases, the student admits it before I finish describing the AST similarity. In the other half, I learn something: a TA gave identical starter code, or the student worked with a lab partner beyond the allowed policy. Sometimes that is still a violation, but the conversation gives me the context a percentage cannot.

In one spring 2023 honor board hearing, a student had changed every identifier, reversed the loop, and renamed the class. The Codequiry AST score was 91%, and the web check traced the original to a tutor's GitHub repository. The student argued they had only referenced the tutor. The board asked one question: why is your redundant if statement identical to the one in the repository? The student did not have an answer. That detail, not the score, ended the discussion.

The goal is to detect code plagiarism accurately, not to maximize referrals. I want students to understand what they submitted, and I want the cases I do send to be supported by evidence a committee can read in one sitting.

Frequently Asked Questions

Can refactored code plagiarism be detected if every variable is renamed?

Yes. Renaming variables changes identifiers, not control-flow structure. Token normalization strips names while preserving operators and statement boundaries, and AST comparison compares the program's structure. A student can fool a character-based diff this way, but not a combined token and AST check.

What peer similarity percentage should trigger an investigation?

There is no universal cutoff. For a 40-line assignment, I start reviewing at 32% peer similarity because I would rather spend five minutes on a false positive than miss a copied lab. For a 300-line project, I raise the threshold to 45% because more boilerplate overlaps by chance.

Does Codequiry work for languages other than Java?

Yes. The comparison engine supports more than 20 programming languages, including Python, C, C++, JavaScript, and Go. My Java-based workflow applies to any course. The tokenizer and AST rules change, but the review process does not.

When you are ready to run the full flow, upload your next assignment folder to Codequiry's peer and web similarity checker.