curator-research

Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-OFF

README

License: apache-2.0

Highlights

Built with Llama: trained on Llama 3.3 70B outputs; architecture Qwen/Qwen3-8B.

Structured extraction of entities, facts and citations from documents.

On 48 held-out documents: valid JSON 48/48, schema 0.945 and grounding 0.858, the best grounding of the three thinking-off enrichment models on the same documents.
Fine-tuning gain over its base on the 120-document selection slice: semantic F1 0.344 to 0.583 (+69%) and quality 0.610 to 0.716 (+17%), with grounding 0.994 to 0.889.
Against the Llama 3.3 70B teacher on 8 paired documents: schema 0.915 versus 0.872 (5 wins, 3 ties, 0 losses) and semantic F1 0.527 versus 0.572.
Small and efficient: the 4-bit 8B base is 6.1 GB (the 4-bit 70B teacher is 39.8 GB) and serves all seven Curator 8B adapters at once, at 0.56 J per generated token against the teacher's 3.70 (conditions differ; see GPU serving).

Assessment: A strong thinking-off enrichment model: the best grounding among the thinking-off variants and higher schema conformance than the teacher on the paired documents. The teacher comparison covers 8 documents.

Architecture: Qwen/Qwen3-8B · Artifact: adapter · Mode: thinking-off.

Family comparison

Evaluation pools and paired comparisons are described below. This release is shown in bold.

Enrichment evaluation — 1 October 2026

48 held-out documents per measured release.

Table with columns: HF model, Valid JSON, Schema, Grounding, Semantic F1
HF model	Valid JSON	Schema	Grounding	Semantic F1
Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-OFF	48/48	.945	.858	.499
Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-ON	48/48	.970	.877	.559

*35B ON semantic F1 covers 47 scored documents; one was unscored. Nemotron ON is not yet evaluated: no held-out case could be proven unseen by this checkpoint.

Paired comparison — 8 documents per release

Values show student / Llama 3.3 70B teacher.

Table with columns: HF model, Schema, Semantic F1, Judge
HF model	Schema	Semantic F1	Judge
Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-ON	.989 / .933	.508 / .571	.750 / .781
Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-OFF	.915 / .872	.527 / .572	.727 / .852
Llama-3.3-Nemotron-Cascade-2-30B-A3B-Curator-0.1.0-Enrichment-Thinking-OFF

The eight-document sample limits generalisation. Schema measures format conformance. References and judge use the teacher family, which can favour the teacher.

Checkpoint-selection comparison (training record)

Base and selected adapter used the same 120-document selection slice; run dates were not preserved.

Table with columns: HF model, Quality: base → adapter, Semantic F1: base → adapter, Grounding: base → adapter
HF model	Quality: base → adapter	Semantic F1: base → adapter	Grounding: base → adapter
Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-OFF	.610 → .716 (+17%)	.344 → .583 (+69%)	.994 → .889
Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-ON	.610 → .736 (+21%)	.344 → .629 (+83%)	.994 → .839
Llama-3.3-Qwen3.6-35B-A3B-Curator-0.1.0-Enrichment-Thinking-ON

Percentages are relative changes calculated from rounded scores. The 35B OFF checkpoint is recorded as a baseline selection. A 3 October check shows the adapter does change the base: on a fixed prompt the base's most likely first token ends the reply, while the adapter's opens the JSON answer.

GPU serving — 1 October 2026

One NVIDIA A100 80 GB, vLLM 0.29. Each row describes a shared serving package.

Table with columns: Package / representative HF model, Base weights, Single stream, Concurrent throughput, GPU J/token, KV capacity (tokens)
Package / representative HF model	Base weights	Single stream	Concurrent throughput	GPU J/token	KV capacity (tokens)
Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-ON + 6 sibling adapters (AWQ)	6.1 GB	111 tok/s	684 tok/s at 32	.56	471,200
Llama-3.3-Qwen3.6-35B-A3B-Curator-0.1.0-Enrichment-Thinking-ON + 3 sibling adapters (W4A16)	21.8 GB

Student sweeps used 2,048 input / 256 output tokens and cold cache. The teacher figure comes from a 180-second warm window with different output lengths. Ratios between these rows combine model and workload effects. Energy covers sampled GPU board power. The 8B base occupies about one-sixth of the teacher's weight storage and serves seven adapters. Nemotron ON throughput remains unmeasured.

Selection measurements (training record)

Recorded result: quality 0.716; semantic F1 0.583; schema 0.903; grounding 0.889.

Comparison pool: 120 documents.

Base on the same pool: Qwen3-8B base on the same slice: quality 0.610; semantic F1 0.344; schema 0.905; grounding 0.994.

From the training record; run dates were not preserved.

Loading and use

Load Qwen/Qwen3-8B with a compatible PEFT runtime and attach this adapter. The measured weight files match the published weight files; the immutable public loading pin is shown below.

Full-precision PEFT loading example, with a pinned public base. The reported 8B serving measurements used the 4-bit AWQ package identified in the tables.

python

from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B", revision="b968826d9c46dd6066d109eabc6255188de91218")base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", revision="b968826d9c46dd6066d109eabc6255188de91218", torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(base, "curator-research/Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-OFF", revision="55232ef34fefd70c78c94e195343bed912f29103")

Shared-base multi-LoRA

A single compatible Qwen/Qwen3-8B base can serve this family of 7 adapters. Register each immutable adapter snapshot with --enable-lora --lora-modules alias=./adapter, then select its alias through the HTTP request model field. This reuses the base weights for fast task switching and GPU-memory reuse. Size the LoRA slots, context and concurrency for the deployment. The 8B and 35B families each require their own base.

Set the chat template's enable_thinking option to false for this variant. The base templates think by default, and an adapter alias does not change the template. The 1 October results used Curator's training-time prompt format; this chat-template setting was not separately evaluated.

Curator publishes these weights as an experimental release of its applied model research, with its own measurements on held-out data. Training data, prompts and the output schema remain private, so the published scores cannot be reproduced outside Curator; evaluate this exact revision on your own data before production use.

License and lineage

Released under Apache-2.0, the base's licence (LICENSE, NOTICE.txt). Because its training targets came from Llama 3.3 70B, the Llama 3.3 Community License attribution and naming terms and the Llama Acceptable Use Policy also apply (TEACHER_LICENSE.txt). Both apply together; this is not a dual licence.

Curator Research

Read the model overview and evaluation boundaries, explore the Curator Research organisation, and follow Curator Research on LinkedIn.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.