The API Bill Your Relation Extraction Pipeline Doesn't Need to Pay
Your document intelligence team is sending entity pairs to GPT-5.4 or Claude Sonnet 4.6. The model returns a relation label. The bill arrives. The throughput is acceptable. Nobody is asking whether a 494-million-parameter model running on a single GPU in your own data center could do the same job better, for a fraction of the cost, without the data leaving your network.
They should be asking.
Relation extraction sits at the core of knowledge graph construction, structured search, and entity-level analytics. Get it wrong and your downstream systems propagate noise. Get it right at API scale and you face a different problem: vendor dependency, per-token cost, and a legal department that has opinions about patient records or contract terms transiting a third-party endpoint.
Researchers at Aristotle University of Thessaloniki published a study in June 2026 that directly addresses this. Despina Christou and Grigorios Tsoumakas ran 30 fine-tuned configurations of five small language models, ranging from 360 million to 3 billion parameters, across nine benchmarks covering both general-domain and literary text. They compared every configuration against zero-shot GPT-5.4 and Claude Sonnet 4.6. The result: the best sub-billion model, a Qwen2.5-0.5B fine-tuned on pooled general-domain data, achieved a general-domain average F1 of 0.83, against 0.69 for GPT-5.4 and 0.66 for Claude Sonnet 4.6 under the same minimal zero-shot protocol. On literary text, the gap widened to more than 25 F1 points. The advantage had nothing to do with model architecture and everything to do with task adaptation: a fine-tuned RoBERTa discriminative baseline, which does not use generative decoding at all, also beat both frontier models, confirming the source of the edge.
Why Scale Stopped Being the Deciding Variable for This Task
The standard assumption in enterprise AI procurement has been that bigger models produce better structured outputs. It is a reasonable prior. It is wrong for relation extraction under these conditions.
The paper's cleanest test of this is within the Qwen2.5 family, where the same model architecture exists at 0.5B and 3B parameters. The 6x scale increase produced an overall F1 gain of only 0.037 on relation extraction, with a 95% confidence interval of +0.009 to +0.067. On the general-domain fine-tuning regime specifically, the gain was negative: -0.004. The 0.5B Qwen2.5 fine-tuned with two-shot prompting scores 0.828 on the seven-benchmark general average; the Llama-3.2-3B under the same conditions scores 0.844. The difference is statistically indistinguishable from noise on general RE when directly comparing Qwen-0.5B GenTune 2-shot against Llama-3B MixTune 2-shot (p = 0.40).
The authors are careful about the cross-family SmolLM comparison, which shows a larger scale effect, because it confounds scale with generational model improvements. The Qwen within-family result is the trustworthy number. A 6x parameter increase buys you 0.037 average F1 on this task.
What does buy you performance is the combination of three things the paper tests systematically:
- Domain composition of training data: Whether you train on general data only, literary data only, or a balanced 50/50 mix. The mixed regime comes within 2 to 3 F1 points of each specialist on its home domain, with a domain-balance gap of only 0.010, meaning you can deploy one model across both contexts without meaningful sacrifice.
- Prompt-conditioned tuning style: Whether in-context examples are present during fine-tuning and inference. The decomposition here is the paper's most operationally important finding: the gains from two-shot configurations come almost entirely from inference-time demonstrations, not from training-time demonstrations. Training-time demonstrations actually hurt sub-billion models by -0.274 F1 on average before inference-time demonstrations are added back. You are not teaching the task during fine-tuning when you include examples. You are teaching the model to depend on examples at inference.
- Which model you start from: Llama-3.2-3B achieves the highest general-domain average at 0.844. SmolLM3-3B achieves the highest literary average at 0.833. Qwen2.5-0.5B is the best sub-billion option at 0.828 general average and fits in approximately 0.5 GB of 4-bit memory.
What Schema Enumeration Is Doing to Your Output Quality
Most practitioners building structured extraction pipelines do the intuitive thing: they include the allowed label set in the system prompt. Tell the model what outputs are valid. Constrain the space. Get cleaner results.
This paper tests that assumption directly and finds it backwards.
Generic prompting, without label enumeration, outperforms schema-enumerated prompting by +3.2 F1 points on average across all nine benchmarks, improving on 8 of 9 datasets. On GIDS specifically, the gap is +0.129. On TACRED it is +0.053. The schema-valid output rate under generic prompting (0.875) is essentially identical to the rate under schema-enumerated prompting (0.882). The model is not producing more invalid outputs when you remove the label list. It is producing better ones.
The likely mechanism, which the paper does not fully resolve, is that enumerating a large label set in the prompt consumes context budget and introduces a form of anchoring that degrades the model's relational reasoning. The exception is CoNLL04, where schema-enumerated prompting edges out generic by -0.015, but CoNLL04 is a type-determined task where entity-type pairs essentially determine the relation, a near-ceiling benchmark that is not measuring the same thing as TACRED or Re-DocRED.
For deployment: remove the label list from your system prompt and let the output normalization layer handle matching. The paper uses exact match after lowercasing and whitespace normalization. Malformed output rates were below 0.1% under both conditions.
The Literary Gap That Reveals Where Frontier Models Actually Break Down
The general-domain results are striking. The literary results are more so, and they point to something specific about where frontier zero-shot reasoning fails.
On the PG-Fiction benchmark, which tests relation extraction from Project Gutenberg fiction using a 137-label ontology, the best tuned SLM scores 0.749 micro-F1. GPT-5.4 scores 0.324. Claude Sonnet 4.6 scores 0.334. The gap on PG-Fiction alone is approximately 0.425 F1 points, larger than the total F1 score either frontier model achieves on this benchmark. This is not a marginal quality difference. It is the difference between a system that works and one that does not.
The frontier models' difficulty with literary text reflects what the paper identifies as the task's structural challenges: implicit semantics, figurative language, long-range dependencies across chapter-length passages, and narrative relations that require tracking character state over time. A zero-shot frontier model applying its general relational reasoning to Victorian prose is not doing what a fine-tuned model trained on labeled literary relation examples does. It is doing something categorically different and less useful.
The validity caveat here matters: PG-Fiction annotations were generated by GPT-4, so tuned SLMs are partly learning to replicate a frontier annotator's distribution. The authors acknowledge this directly. The cleaner check is the human-annotated Biographical benchmark, where the best SLM scores 0.917 versus GPT-5.4's 0.832, a gap of approximately 8 F1 points that survives the paper's data-leakage analysis. About 23% of Biographical test examples appear verbatim in the training split due to per-example rather than per-document splitting. When the authors re-score on the de-leaked test set, the SLM margin over GPT-5.4 is essentially unchanged.
Why Domain-Adaptive Pretraining Did Not Move the Needle
The paper runs a separate case study on domain-adaptive pretraining, the practice of continuing language model pre-training on unlabeled in-domain text before supervised fine-tuning. This approach has a strong track record in biomedical NLP (BioBERT, SciBERT) and was a reasonable hypothesis for improving literary domain performance.
The result is a null: DAPT on approximately 80 million tokens of LitBank text produced at most 0.001 average literary F1 improvement for Llama-3.2-3B. The difference on PG-Fiction under LitTune was 0.740 to 0.742. This is a negative finding in the productive sense: it tells teams where not to invest time and compute.
The authors note this may reflect that modern instruction-tuned base models have already absorbed sufficient literary text during pre-training, leaving little room for in-distribution text adaptation to add signal. It may also reflect that the bottleneck for literary RE is labeled supervision for the specific relation schema, not general literary language familiarity. A 3B model trained on 80 million tokens of fiction still knows what a novel sounds like. What it does not know, until you show it, is which entity pairs carry which relation labels under your schema.
The practical implication: if you are standing up a relation extraction pipeline for a specialized domain, budget for labeled data collection and fine-tuning. Do not budget for continued pre-training unless you have a strong prior reason to believe the base model genuinely lacks domain vocabulary.
A Clinical NLP Team That Cannot Afford API Exposure
Consider the deployment context this paper's results are most directly built for: a healthcare system running clinical document processing that cannot route patient data to a third-party API. The team needs relation extraction at volume from discharge summaries, physician notes, and referral letters, connecting entities like diagnoses, medications, procedures, and patient identifiers.
The current architecture, zero-shot via frontier API, has a validity problem beyond cost. Every inference call transmits protected health information to a vendor endpoint. Legal says no. IT security says no. The compliance officer says no.
With the approach this paper validates, the team fine-tunes Qwen2.5-0.5B on their labeled RE dataset using QLoRA on a single RTX 4090. Training time is approximately 16 to 22 hours. The resulting checkpoint is approximately 0.5 GB at 4-bit. Inference runs at approximately 22 milliseconds per example on GPU or approximately 180 milliseconds on a CPU server, with no external network call. The model fits entirely in local infrastructure.
The 0.5B model operating locally not only eliminates the compliance exposure, it outperforms the frontier model it replaces by approximately 0.13 to 0.17 F1 points on general relation extraction, a gap that compounds across millions of document pairs. The deployment also supports CPU inference via llama.cpp at Q4_K_M quantization, which matters for teams whose GPU availability is limited or whose inference workload is episodic rather than batch-scale.
The one caveat the authors note explicitly: local execution does not by itself guarantee privacy. Logging, disk security, and telemetry configuration still require attention. This is an infrastructure hygiene point, not a disqualification of the approach.
What Your Procurement Assumptions Are Costing You on Structured Extraction
The paper's findings create a specific decision point for organizations running or planning relation extraction workloads. Not a general AI modernization question. A specific one: are you paying frontier API rates and accepting frontier accuracy on a task where a 494-million-parameter local model demonstrably performs better?
The 30-configuration study gives practitioners a deployment map rather than a single recommendation. If your workload is general-domain only and throughput matters above all, Qwen2.5-0.5B with GenTune and two-shot prompting at 0.828 F1 is the efficiency-optimal choice: approximately 0.5 GB, approximately 22 milliseconds per inference, no API dependency. If your workload spans general and literary or mixed document types, the MixTune regime costs you at most 2 to 3 F1 points against the specialist configuration in each domain and eliminates the need to route documents to different models. If your target domain is literary or narrative text, the 3B models under LitTune configurations lead the best 0.5B literary scores by approximately 5 F1 points, which may or may not justify the 4x parameter cost depending on your accuracy floor.
The scale-versus-adaptation finding has implications beyond this paper's specific benchmarks. If a 6x parameter increase within the same model family produces 0.037 average F1 gain on this task, the engineering investment that justifies the next scale tier is far higher than most procurement frameworks assume. The question is not whether your model is large enough. The question is whether your fine-tuning data and training strategy are well enough specified.
Frontier models are not getting worse. Their zero-shot relation extraction will improve. But the structural advantage of task-specific adaptation does not evaporate as base models scale: the discriminative RoBERTa baseline also beat both frontier models in this study, and RoBERTa-large is 355 million parameters from 2019. The gap between a well-adapted small model and a well-prompted large model on specialized extraction tasks is not closing on the timeline most organizations are budgeting against.
The checkpoint is already on Hugging Face. The training code is already on GitHub. The only question is whether you are running the experiment.
Agents Applied covers the applied AI research that changes how organizations build and deploy intelligent systems.