curator-research

Llama-3.3-Qwen3-8B-Curator-0.1.0-Synthesis-OFF

README

License: apache-2.0

Highlights

Built with Llama: trained on Llama 3.3 70B outputs; architecture Qwen/Qwen3-8B.

Grounded answers from supplied passages, with citations and abstention.

Highest citation recall in the family: 0.960 on the 118 questions with reference citations, against the teacher's 0.875; refusal agreement 139/139, including all 21 questions that call for a refusal.
It does not stop on its own (0/139 natural stops), so serve it with an output limit. An end-of-answer fine-tune fixed stopping (139/139) but lowered citation recall (0.957 to 0.796 on the BF16 base), so this release keeps the higher-recall weights.

Assessment: High-recall grounded-answer adapter that matches every refusal case; serve it with an output limit, since its long answers cite more passages than needed (citation precision 0.571 on the BF16 base).

Architecture: Qwen/Qwen3-8B · Artifact: adapter · Mode: off.

Family comparison

Evaluation pools and paired comparisons are described below. This release is shown in bold.

Synthesis evaluation — 1 October 2026

Table with columns: HF model, Natural stop /139, Raw gate /139, 9-sentence gate /139, Citation recall, Refusal agreement /139
HF model	Natural stop /139	Raw gate /139	9-sentence gate /139	Citation recall	Refusal agreement /139
Llama-3.3-Qwen3-8B-Curator-0.1.0-Synthesis-OFF	0	14	82	.960	139
Llama-3.3-Qwen3-8B-Curator-0.1.0-Synthesis-THINK	116	74	92	.918	135
Llama 3.3 70B AWQ	138	96	87

Citation recall uses the 118 questions with cited references and the raw outputs. The nine-sentence policy applies to the gate column only; recall after cutting outputs was not measured here. Refusal agreement counts all 139 cases; 21 of them call for a refusal. Citation precision on the BF16 base (1 October): OFF .571, citing 5.0 passages per answer against 2.4 in the references; THINK .720.

Table with columns: HF model, Blind judge: student / teacher, Scored pairs
HF model	Blind judge: student / teacher	Scored pairs
Llama-3.3-Qwen3-8B-Curator-0.1.0-Synthesis-OFF	.795 / .917	131
Llama-3.3-Qwen3-8B-Curator-0.1.0-Synthesis-THINK	.839 / .921	131

Twelve judge items are missing across the three model groups. The judge uses the teacher's model family. Against the teacher, THINK has a higher 9-sentence gate count (92 vs 87) and raw citation recall (.918 vs .875); its raw gate (74 vs 96), natural stops (116 vs 138) and judge score are lower. OFF does not stop on its own: 126 outputs hit the token limit and 13 the repetition guard. A later end-of-answer fine-tune fixed stopping (139/139) but lowered citation recall (.957 → .796 on the BF16 base), so it was not released.

Table with columns: HF model, Turn type /48, Natural stop /48, Paired student / teacher
HF model	Turn type /48	Natural stop /48	Paired student / teacher
Llama-3.3-Qwen3-8B-Curator-0.1.0-Synthesis-AGENTIC	40/48	48/48	15/17 / 3/17

AGENTIC's pool has 41 thought and 7 final turns and no tool calls; always answering 'thought' would score 41/48. The adapter finds 4 of the 7 final turns. The paired result measures format alignment; tool calls were not part of this evaluation.

GPU serving — 1 October 2026

One NVIDIA A100 80 GB, vLLM 0.29. Each row describes a shared serving package.

Table with columns: Package / representative HF model, Base weights, Single stream, Concurrent throughput, GPU J/token, KV capacity (tokens)
Package / representative HF model	Base weights	Single stream	Concurrent throughput	GPU J/token	KV capacity (tokens)
Llama-3.3-Qwen3-8B-Curator-0.1.0-Enrichment-Thinking-ON + 6 sibling adapters (AWQ)	6.1 GB	111 tok/s	684 tok/s at 32	.56	471,200
Llama-3.3-Qwen3.6-35B-A3B-Curator-0.1.0-Enrichment-Thinking-ON + 3 sibling adapters (W4A16)	21.8 GB

Student sweeps used 2,048 input / 256 output tokens and cold cache. The teacher figure comes from a 180-second warm window with different output lengths. Ratios between these rows combine model and workload effects. Energy covers sampled GPU board power. The 8B base occupies about one-sixth of the teacher's weight storage and serves seven adapters. Nemotron ON throughput remains unmeasured.

Loading and use

Load Qwen/Qwen3-8B with a compatible PEFT runtime and attach this adapter. The measured weight files match the published weight files; the immutable public loading pin is shown below.

Full-precision PEFT loading example, with a pinned public base. The reported 8B serving measurements used the 4-bit AWQ package identified in the tables.

python

from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B", revision="b968826d9c46dd6066d109eabc6255188de91218")base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", revision="b968826d9c46dd6066d109eabc6255188de91218", torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(base, "curator-research/Llama-3.3-Qwen3-8B-Curator-0.1.0-Synthesis-OFF", revision="9e03b4f39f89948457a050de864712d570bd73bd")

Shared-base multi-LoRA

A single compatible Qwen/Qwen3-8B base can serve this family of 7 adapters. Register each immutable adapter snapshot with --enable-lora --lora-modules alias=./adapter, then select its alias through the HTTP request model field. This reuses the base weights for fast task switching and GPU-memory reuse. Size the LoRA slots, context and concurrency for the deployment. The 8B and 35B families each require their own base.

Set the chat template's enable_thinking option to false for this variant. The base templates think by default, and an adapter alias does not change the template. The 1 October results used Curator's training-time prompt format; this chat-template setting was not separately evaluated.

Curator publishes these weights as an experimental release of its applied model research, with its own measurements on held-out data. Training data and prompts remain private, so the published scores cannot be reproduced outside Curator; evaluate this exact revision on your own data before production use.

License and lineage

Released under Apache-2.0, the base's licence (LICENSE, NOTICE.txt). Because its training targets came from Llama 3.3 70B, the Llama 3.3 Community License attribution and naming terms and the Llama Acceptable Use Policy also apply (TEACHER_LICENSE.txt). Both apply together; this is not a dual licence.

Curator Research

Read the model overview and evaluation boundaries, explore the Curator Research organisation, and follow Curator Research on LinkedIn.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

curator-research

Model Tree

Base

Qwen/Qwen3-8B

Adapter

this model

Input Modalities

Text

Output Modalities