Status and lineage
This adapter is the canonical Qwen 2.5 3B artifact used by the Brie paper and
evaluation pipeline. closestfriend/brie-v2-qwen2.5-3b is a later full-weight
distribution produced by merging this adapter into the base model; it is not a
separate result.
Evaluation
The comprehensive evaluation contains 57 judgments over 25 unique prompts
(none of which appear in the training data); configuration blocks repeat
prompts.
Judges received blind, position-randomized A/B comparisons of the same base and
Brie responses. Primary rates retain ties and assigned prompts in the
denominator.
Table with columns: Panel, Brie preference, Construction| Panel | Brie preference | Construction |
|---|
| 2025 primary | 88.9% | Mean of three full n=57 passes: Claude Opus 4, GPT-4o, and Gemini 2.5 Flash Lite |
| 2025 expanded sensitivity | 90.5% | Adds Claude 3.5 Sonnet (n=42) to the three full passes; prompt coverage is unequal |
| 2026 primary | 78.9% | Four-model mean, n=57 per model: Claude Haiku 4.5, Claude Sonnet 4.5, GPT-5, and Gemini 3 Pro |
The historical 91.2% result is the original split-source Anthropic pool:
Claude 3.5 Sonnet judged 42 prompts and Claude Opus 4 judged the other 15,
yielding 52/57 Brie preferences. It is not a uniform multi-judge mean. A
separate GPT-4o/Gemini verdict-agreement calculation also happens to equal
91.2% (52/57).
The small n=15 out-of-domain control provides no detected broad regression
under the tested panels, but is too small to establish absence of capability
loss.
Full pass identities, counts, hashes, and reporting decisions are documented
in the paper repository's docs/QWEN_3B_2025_JUDGE_PROVENANCE.md and generated
claim ledger.
Training details
- Base model:
Qwen/Qwen2.5-3B-Instruct
- Method: LoRA, rank 16, alpha 32, dropout 0.05
- Target modules:
q_proj, v_proj
- Training corpus: the 1,153-example snapshot of a corpus authored by the
researcher with LLMs used as discussion and authoring tools (595 unique
prompt–response pairs across 202 prompts; see the paper's Section 7.2)
- Training: configured for 2 epochs (290 steps) with best-checkpoint
selection; this published adapter is checkpoint-200 (epoch ≈1.38), the
best-validation-loss checkpoint, and is the adapter the paper evaluated
- Language: English
Use
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_id = "Qwen/Qwen2.5-3B-Instruct"adapter_id = "closestfriend/brie-v2-3b" tokenizer = AutoTokenizer.from_pretrained(base_id)base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")model = PeftModel.from_pretrained(base, adapter_id)
This is a research artifact. The corpus reflects one author's English-language
philosophical and creative practice; it is not a general factual-authority or
safety-tuned model.
Resources
Citation
@misc{karman_brie_domain_adaptation, author = {Karman, Hunter N.}, title = {Human-Curated Data Authoring with LLMs: A Small-Data Approach to Domain Adaptation}, doi = {10.5281/zenodo.17657737}, url = {https://doi.org/10.5281/zenodo.17657737}}