Sentence-Level AI Detection for Academic Integrity: A Granular Approach to Reducing False Positives in Mixed-Authorship Submissions

Most deployed AI detection systems treat a submission as a single input and return a single probability estimate. This works reasonably well when the document is entirely human-written or entirely machine-generated. The problem is that this is not how students actually use LLMs.

In practice, the common pattern is partial use: a student writes most of an essay themselves, then uses ChatGPT to generate an introduction, polish a few paragraphs, or draft a literature review section. A Coursera/Censuswide survey (fieldwork October 2025, published February 2026) found that 80% of students reported AI tools improved their academic performance, while only 20% of educators reported that their university had a formal AI policy in place. The gap is not surprising. What it means for detection is that the real-world input distribution is dominated by hybrid documents, not pure-class ones.

A document-level classifier receiving a 3,000-word essay with two AI-generated paragraphs faces an awkward choice. If its aggregation function is sensitive, it flags the whole document and the educator sees “68% AI probability” with no way to locate the actual AI content. If it’s conservative, the signal from two paragraphs gets diluted and the system misses it entirely.

Document-Level Classifiers

Most AI detection tools operate at the document level, though several now overlay sentence-level highlighting as a post-hoc visualization step. The underlying classifiers are typically transformers trained on binary classification (human vs. machine) using perplexity, burstiness, and distributional features (Mitchell et al., 2023; Gehrmann et al., 2019). GPTZero reports detecting 95.7% of AI text at a 1% document-level false positive rate on the RAID benchmark, with accuracy exceeding 99% when filtered to modern frontier LLMs. Independent evaluations tell a different story: a 2023 evaluation of 14 detectors found that all scored below 80% accuracy and only 5 exceeded 70%, with a systematic bias toward classifying text as human-written. (Weber-Wulff et al., 2023)

Non-Native Writer Bias

Non-native writers use more predictable vocabulary, shorter sentences, and simpler syntactic constructions. Perplexity-based detectors interpret low perplexity as a machine signal, which creates a direct conflict with the linguistic profile of L2 writers. Vanderbilt disabled Turnitin’s AI detector in August 2023 specifically because of this issue.

Sentence-Level and Sequence Labeling Approaches

Emi and Spero (2024) introduced hard negative mining and synthetic mirror training for reducing false positives in a transformer-based text classifier. Lekkala et al. (2025) framed AI detection as a sentence-level segmentation task using CRF layers on top of transformer encoders. Our work is closest to Lekkala et al. but uses a simpler classification head with a sliding context window, trading global sequence dependencies for faster inference, and focuses specifically on academic use cases with attendant fairness requirements around non-native English writers.

The classifier is built on a proprietary fine-tuned transformer encoder. For each target sentence, we construct an input sequence consisting of the two preceding sentences, a separator, the target sentence, another separator, then the two following sentences. The flanking sentences provide discourse context; the target sentence is the classification focus.

The classification head is a single linear layer over the pooled encoder representation, producing a scalar logit passed through sigmoid to yield a probability that the sentence is AI-generated, given its context. We apply temperature scaling on a held-out calibration set of 5,000 sentences to produce calibrated confidence scores. Post-calibration, a sentence scored at 0.90 is empirically AI-generated 89.4% of the time, within 1% of stated confidence across all bins above 0.5.

After initial fine-tuning, we run the model over a pool of 200k human-authored sentences held out from training. Every sentence the model classifies as AI with probability above 0.5 is extracted as a hard negative and added to the training set at a 1:3 ratio. This cycle repeats five times.

The hard negative pool is deliberately enriched with categories that produce elevated false positive rates in pilot testing: non-native English academic text (TOEFL/IELTS essay corpora, ESL thesis collections), formulaic disciplinary prose (clinical case reports, legal briefs, patent applications), and highly structured formats (methods sections, lab protocols, statistical results paragraphs). Without hard negative mining, the model treats low-perplexity, predictable prose as an AI signal regardless of authorship. The mining forces it to distinguish between “predictable because formulaic” and “predictable because machine-generated.”

For each human document, we also generate a synthetic mirror: an AI-generated text matched in topic, structure, length, and approximate register. This prevents the model from learning spurious correlations between subject matter and class labels. Emi and Spero (2024) demonstrated the effectiveness of this approach for general-domain text; we extend it to academic writing with format-aware prompting.

The evaluation set is disjoint from training – no overlapping source documents or authors where attribution was available. It consists of three subsets: 4,200 fully human-authored academic texts (stratified by discipline, document length, and writer demographics, with a 10% random sample manually verified by two annotators at Cohen’s kappa = 0.94); 3,800 fully AI-generated texts from all eight LLMs including the two blind-tested models; and 2,600 mixed-authorship documents with sentence-level ground truth.

The non-native English subset (n=620) was drawn from the human-only pool and includes texts from writers whose first language spans 23 language families. We report document-level false positive rate, sentence-level precision/recall/F1 on the mixed-authorship subset, FPR disaggregated by native vs. non-native writer populations, and per-model detection accuracy. We compare against a document-level transformer classifier trained on the same data and DetectGPT applied at the sentence level. All baselines use the 0.85 confidence threshold for FPR calculation.

The main result is that sentence-level classification with hard negative mining dramatically reduces the FPR gap between native and non-native English writers. This matters for institutional deployment. A university that processes submissions from a diverse student body cannot accept a system that disproportionately flags international students; beyond the fairness concern, it creates real legal exposure.

The sentence-level output also changes how the tool gets used in practice. When an educator sees a single document score of 68%, they face a binary decision with no supporting evidence. When they see four highlighted sentences in an otherwise clean document, they can point to specific passages and ask the student about them. This shifts the workflow from “the tool says you cheated” to “can you walk me through how you wrote these sections?” – which is closer to how academic integrity conversations should work.

Sentence-level granularity is not just a technical improvement over document-level classification – it changes the operational model for how detection tools are used in academic settings. When the output is specific and interpretable, the tool supports evidence-based conversations instead of binary verdicts.

For institutions deploying AI detection at scale, the practical implications are concrete: dramatically lower false positive rates on non-native English writing, the ability to point to specific flagged passages instead of document-wide scores, and a workflow that fits how academic integrity conversations should actually happen. The full methodology and results are available in the research paper (PDF).