barozp

Qwen3.8-27B-Opus-Distill-v2

README

License: apache-2.0

Why this release exists

A community member (zxbc2023) reported a clean, deterministic bug on v1: certain stacked output-format constraints (most notably "no prose" combined with a second constraint like

markdown

"no markdown"

) sent the model into a non-converging self-verification loop — burning the entire token budget on repeated "wait, let me re-check…" passes with zero visible output. Their repro was exact and reproducible at temp=0, and they correctly diagnosed it as likely inherited from the distillation data rather than a capability gap. Full credit to them for the report that made this fix possible.

Root cause. Investigating v1's training data (opus-reasoning-distill-train) showed that 83% of it had actually been sourced from Jackrong's TraceInversion datasets — reasoning traces reconstructed by a small auxiliary model (Trace-Inverter-4B) from a compressed summary, not the teacher model's real output (openly documented on those datasets' own cards). That reconstruction process appears to have baked in a rigid "enumerate constraints, then re-verify" template — plausibly both the source of v1's reasoning-quality gains and the loop bug, two sides of the same mechanism.

The fix, in two steps:

First attempt: retrain purely on a genuine-trace dataset (lordx64/reasoning-distill-claude-opus-4-7-max). This fixed the loop bug completely but collapsed the reasoning-transfer gain (GPQA-Diamond dropped back to base/random level) — the domain mix mattered as much as trace authenticity.
This release: reconstructed v1's exact prompt distribution, but replaced every row traced back to Jackrong's fabricated sets with the verified genuine trace for the same prompt (from lordx64 or Roman1111111/claude-opus-4.6-10000x). Then ran a full-dataset scan (30 parallel LLM-judge passes over all 14,250 rows) for the specific pathological pattern and removed every flagged row (169) plus every row whose source was never verified against a known-genuine dataset (2,429). Net: 14,250 → 11,716 rows, 100% traced to verified sources.

Bug fix validation

Re-ran zxbc2023's exact repro (temp=0, seed=1234, fib(n) with stacked constraints) against this model:

Table with columns: Test, v1, v2
Test	v1	v2
"no prose" + "no markdown"	3000/3000 tokens, 0 visible output	87/4096 tokens, clean code
"no prose" + "no comments"	3000/3000 tokens, 0 visible output	73/4096 tokens, clean code
Generalization (different task, same constraints)	—	43/4096 tokens, clean code

All three converge quickly with </think> closing normally — no loop, no budget exhaustion.

Benchmark results

Measured with lm-evaluation-harness, 0-shot, loglikelihood (multiple-choice), chat template OFF, QUICK mode (--limit 500) — identical protocol to v1, so the Δ column is the meaningful signal.

Table with columns: Task, Metric, Base, v2, Δ, (v1's Δ, for reference)
Task	Metric	Base	v2	Δ	(v1's Δ, for reference)
wikitext	word perplexity ↓	8.4335	8.3788	−0.055	−0.09
mmlu	acc	0.8494	0.8476	−0.002	−0.001
hellaswag	acc_norm

Reading the table: knowledge (MMLU) and language modeling (wikitext) are unaffected; reasoning (ARC, GPQA) shows a large, real gain — slightly smaller than v1's (expected: v1's gain was partly driven by the same fabricated content that caused the bug), but still far above noise. GPQA in particular is a same-protocol delta vs. base, not comparable to Qwen's own published 89.2 (thinking-mode-on, different harness) — see v1's card for the full caveat.

Model overview

Base model: Qwen/Qwen3.8-27B — dense 27B, native vision-language, native MTP.
Method: LoRA (r=64, alpha=64, dropout=0.05), merged into the base weights.
LoRA targets: attention q/k/v/o_proj on the 16 full-attention layers only; FFN gate/up/down_proj on all 64 layers.
Training data: barozp/opus-reasoning-distill-v2 (11,716 examples, 100% verified-genuine Opus traces).
Training run: 1 epoch (696 steps), lr=1e-4 cosine with 3% warmup, effective batch 16, MAX_SEQ=4096, bf16, ~4h15m on an A100 80GB.
— lower than both v1 (0.4647) and the intermediate pure-genuine attempt (0.805), despite less data.

Quick start

python
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "barozp/Qwen3.8-27B-Opus-Distill-v2", dtype=torch.bfloat16, device_map="auto",
)
processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill-v2")

Text-only inference also works via AutoModelForCausalLM.

Qwen3.8-27B — the base model.
barozp/Qwen3.8-27B-Opus-Distill — v1. Still works well for most use cases; this release specifically fixes the stacked-constraint reasoning-loop edge case.
barozp/opus-reasoning-distill-v2 — the training dataset, with full provenance notes.
Thanks to zxbc2023 for the bug report that led to this release.

Quantized releases of this checkpoint:

barozp/Qwen3.8-27B-Opus-Distill-v2-FP8 -- block-wise FP8 (e4m3, 128x128) for CUDA serving stacks; paired BF16-vs-FP8 evals show deltas within noise.
barozp/Qwen3.8-27B-Opus-Distill-v2-GGUF -- llama.cpp / Ollama / LM Studio quants (BF16 down to IQ1_M), mmproj vision, MTP self-speculative decoding.
barozp/Qwen3.8-27B-Opus-Distill-v2-MLX-4bit and MLX-8bit -- Apple Silicon mlx-vlm conversions.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

barozp

Model Tree

Base

Qwen/Qwen3.8-27B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Why this release exists

A community member (zxbc2023) reported a clean, deterministic bug on v1: certain stacked output-format constraints (most notably "no prose" combined with a second constraint like

markdown

"no markdown"

The fix, in two steps:

First attempt: retrain purely on a genuine-trace dataset (lordx64/reasoning-distill-claude-opus-4-7-max). This fixed the loop bug completely but collapsed the reasoning-transfer gain (GPQA-Diamond dropped back to base/random level) — the domain mix mattered as much as trace authenticity.
This release: reconstructed v1's exact prompt distribution, but replaced every row traced back to Jackrong's fabricated sets with the verified genuine trace for the same prompt (from lordx64 or Roman1111111/claude-opus-4.6-10000x). Then ran a full-dataset scan (30 parallel LLM-judge passes over all 14,250 rows) for the specific pathological pattern and removed every flagged row (169) plus every row whose source was never verified against a known-genuine dataset (2,429). Net: 14,250 → 11,716 rows, 100% traced to verified sources.

Bug fix validation

Re-ran zxbc2023's exact repro (temp=0, seed=1234, fib(n) with stacked constraints) against this model:

Table with columns: Test, v1, v2
Test	v1	v2
"no prose" + "no markdown"	3000/3000 tokens, 0 visible output	87/4096 tokens, clean code
"no prose" + "no comments"	3000/3000 tokens, 0 visible output	73/4096 tokens, clean code
Generalization (different task, same constraints)	—	43/4096 tokens, clean code

All three converge quickly with </think> closing normally — no loop, no budget exhaustion.

Benchmark results

Table with columns: Task, Metric, Base, v2, Δ, (v1's Δ, for reference)
Task	Metric	Base	v2	Δ	(v1's Δ, for reference)
wikitext	word perplexity ↓	8.4335	8.3788	−0.055	−0.09
mmlu	acc	0.8494	0.8476	−0.002	−0.001
hellaswag	acc_norm

Model overview

Base model: Qwen/Qwen3.8-27B — dense 27B, native vision-language, native MTP.
Method: LoRA (r=64, alpha=64, dropout=0.05), merged into the base weights.
LoRA targets: attention q/k/v/o_proj on the 16 full-attention layers only; FFN gate/up/down_proj on all 64 layers.
Training data: barozp/opus-reasoning-distill-v2 (11,716 examples, 100% verified-genuine Opus traces).
Training run: 1 epoch (696 steps), lr=1e-4 cosine with 3% warmup, effective batch 16, MAX_SEQ=4096, bf16, ~4h15m on an A100 80GB.
— lower than both v1 (0.4647) and the intermediate pure-genuine attempt (0.805), despite less data.

Quick start

python
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "barozp/Qwen3.8-27B-Opus-Distill-v2", dtype=torch.bfloat16, device_map="auto",
)
processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill-v2")

Text-only inference also works via AutoModelForCausalLM.

Qwen3.8-27B — the base model.
barozp/Qwen3.8-27B-Opus-Distill — v1. Still works well for most use cases; this release specifically fixes the stacked-constraint reasoning-loop edge case.
barozp/opus-reasoning-distill-v2 — the training dataset, with full provenance notes.
Thanks to zxbc2023 for the bug report that led to this release.

Quantized releases of this checkpoint:

barozp/Qwen3.8-27B-Opus-Distill-v2-FP8 -- block-wise FP8 (e4m3, 128x128) for CUDA serving stacks; paired BF16-vs-FP8 evals show deltas within noise.
barozp/Qwen3.8-27B-Opus-Distill-v2-GGUF -- llama.cpp / Ollama / LM Studio quants (BF16 down to IQ1_M), mmproj vision, MTP self-speculative decoding.
barozp/Qwen3.8-27B-Opus-Distill-v2-MLX-4bit and MLX-8bit -- Apple Silicon mlx-vlm conversions.

Qwen3.8-27B-Opus-Distill-v2

README

Why this release exists

Bug fix validation

Benchmark results

Model overview

Quick start

Related

Explore FriendliAI today

README

Why this release exists

Bug fix validation

Benchmark results

Model overview

Quick start

Related