barozp

Qwen3.8-27B-Opus-Distill

README

License: apache-2.0

Known issues

Reasoning loop under stacked output-format constraints. Reported by zxbc2023 (full writeup, discussion #1). Combining "no prose" with a second output-format constraint (e.g.

markdown

"no markdown"

or "no comments") can send this model into a non-converging self-verification reasoning loop -- it burns the entire token budget with zero visible output. Fully deterministic and reproducible at temp=0. Root cause: traced to part of the training data being sourced from reconstructed (not verbatim) Opus reasoning traces, not a capability gap.

Fixed in barozp/Qwen3.8-27B-Opus-Distill-v2 -- retrained on a rebuilt dataset where every row is traced to a verified genuine source. If you're hitting this, switch to v2.

Workaround if staying on this version: avoid combining "no prose" with another format constraint, or raise the generation token budget to >=4096 for constrained code-gen tasks.

Model overview

Base model: Qwen/Qwen3.8-27B — dense 27B, native vision-language, native MTP.
Method: LoRA (r=64, alpha=64, dropout=0.05), merged into the base weights.
LoRA targets: attention q/k/v/o_proj on the 16 full-attention layers only; FFN gate/up/down_proj on all 64 layers. Gated-DeltaNet (linear-attention) projections are left untouched, matching the Qwen3.6 recipe.
Training data: barozp/opus-reasoning-distill-train (14,250 examples) + -validation (750, held out).
Training run: 1 epoch (891 steps), lr=1e-4 cosine with 3% warmup, effective batch 16 (micro-batch 1 × grad-accum 16), MAX_SEQ=4096, bf16, ~5h52m on an A100 80GB.
Final validation loss: 0.4647 (step 891/891).
Vision + MTP: carried over from base, never trained.

Highlights

Reasoning improves, nothing else degrades. ARC-Challenge and GPQA-Diamond both rise while MMLU (general knowledge) and wikitext (language modeling) stay flat within noise.
Reflex-level reasoning gets the biggest boost. GPQA is measured in loglikelihood mode with thinking disabled — the model has no chance to deliberate. Distillation more than doubles that "reflex" score, meaning the model internalized step-by-step reasoning rather than just learning to generate <think> blocks.
Vision and MTP are fully intact. This is a complete multimodal model with self-speculative decoding, not a text-only strip-down.

Benchmark results

Measured with lm-evaluation-harness, 0-shot, loglikelihood (multiple-choice), chat template OFF, QUICK mode (--limit 500). Base and distill were run with the identical harness, so the Δ column is the meaningful signal — not the absolute values.

Table with columns: Task, Metric, Base, Distill, Δ
Task	Metric	Base	Distill	Δ
wikitext	word perplexity ↓	8.434	8.344	−0.09
mmlu	acc	0.849	0.849	−0.001
hellaswag	acc_norm	0.742	0.740	−0.002
arc_challenge

Important caveats

GPQA is not comparable to Qwen's published 89.2. Qwen reports GPQA with thinking mode ON and their own harness. Here GPQA is measured in loglikelihood mode with thinking OFF — the base model scores near random (25%) precisely because it gets no chance to reason. The +26pt Δ is a valid same-protocol comparison (base vs distill), but do not place 0.495 next to Qwen's 89.2 as if they were the same measurement.
ARC-Challenge is saturated for modern models. The +4.2pt Δ is real and consistent with the Qwen3.6 release, but ARC (2018, middle-school science) is not a headline reasoning benchmark anymore. GPQA is the stronger signal here.
QUICK mode limits each task to 500 samples; hellaswag and arc_challenge are noisy at that size. Treat small Δ values (±0.01) on those as noise.

Quick start

python
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "barozp/Qwen3.8-27B-Opus-Distill", dtype=torch.bfloat16, device_map="auto",
)
processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill")

Text-only inference also works via AutoModelForCausalLM (vision is ignored, which is the expected/standard behavior for that class).

Qwen3.8-27B — the base model.
barozp/Qwen3.8-27B-Opus-Distill-LoRA-Adapter — the raw LoRA adapter (private; the merged weights here are the same thing applied).
barozp/Qwen3.8-27B-Opus-Distill-GGUF — GGUF quantizations, BF16 down to IQ1_M, imatrix-calibrated, includes the vision mmproj file.
barozp/Qwen3.8-27B-Opus-Distill-v2 — bugfix release, see Known Issues above.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

barozp

Model Tree

Base

Qwen/Qwen3.8-27B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Known issues

Reasoning loop under stacked output-format constraints. Reported by zxbc2023 (full writeup, discussion #1). Combining "no prose" with a second output-format constraint (e.g.

markdown

"no markdown"

Fixed in barozp/Qwen3.8-27B-Opus-Distill-v2 -- retrained on a rebuilt dataset where every row is traced to a verified genuine source. If you're hitting this, switch to v2.

Workaround if staying on this version: avoid combining "no prose" with another format constraint, or raise the generation token budget to >=4096 for constrained code-gen tasks.

Model overview

Base model: Qwen/Qwen3.8-27B — dense 27B, native vision-language, native MTP.
Method: LoRA (r=64, alpha=64, dropout=0.05), merged into the base weights.
LoRA targets: attention q/k/v/o_proj on the 16 full-attention layers only; FFN gate/up/down_proj on all 64 layers. Gated-DeltaNet (linear-attention) projections are left untouched, matching the Qwen3.6 recipe.
Training data: barozp/opus-reasoning-distill-train (14,250 examples) + -validation (750, held out).
Training run: 1 epoch (891 steps), lr=1e-4 cosine with 3% warmup, effective batch 16 (micro-batch 1 × grad-accum 16), MAX_SEQ=4096, bf16, ~5h52m on an A100 80GB.
Final validation loss: 0.4647 (step 891/891).
Vision + MTP: carried over from base, never trained.

Highlights

Reasoning improves, nothing else degrades. ARC-Challenge and GPQA-Diamond both rise while MMLU (general knowledge) and wikitext (language modeling) stay flat within noise.
Reflex-level reasoning gets the biggest boost. GPQA is measured in loglikelihood mode with thinking disabled — the model has no chance to deliberate. Distillation more than doubles that "reflex" score, meaning the model internalized step-by-step reasoning rather than just learning to generate <think> blocks.
Vision and MTP are fully intact. This is a complete multimodal model with self-speculative decoding, not a text-only strip-down.

Benchmark results

Table with columns: Task, Metric, Base, Distill, Δ
Task	Metric	Base	Distill	Δ
wikitext	word perplexity ↓	8.434	8.344	−0.09
mmlu	acc	0.849	0.849	−0.001
hellaswag	acc_norm	0.742	0.740	−0.002
arc_challenge

Important caveats

GPQA is not comparable to Qwen's published 89.2. Qwen reports GPQA with thinking mode ON and their own harness. Here GPQA is measured in loglikelihood mode with thinking OFF — the base model scores near random (25%) precisely because it gets no chance to reason. The +26pt Δ is a valid same-protocol comparison (base vs distill), but do not place 0.495 next to Qwen's 89.2 as if they were the same measurement.
ARC-Challenge is saturated for modern models. The +4.2pt Δ is real and consistent with the Qwen3.6 release, but ARC (2018, middle-school science) is not a headline reasoning benchmark anymore. GPQA is the stronger signal here.
QUICK mode limits each task to 500 samples; hellaswag and arc_challenge are noisy at that size. Treat small Δ values (±0.01) on those as noise.

Quick start

python
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "barozp/Qwen3.8-27B-Opus-Distill", dtype=torch.bfloat16, device_map="auto",
)
processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill")

Text-only inference also works via AutoModelForCausalLM (vision is ignored, which is the expected/standard behavior for that class).

Qwen3.8-27B — the base model.
barozp/Qwen3.8-27B-Opus-Distill-LoRA-Adapter — the raw LoRA adapter (private; the merged weights here are the same thing applied).
barozp/Qwen3.8-27B-Opus-Distill-GGUF — GGUF quantizations, BF16 down to IQ1_M, imatrix-calibrated, includes the vision mmproj file.
barozp/Qwen3.8-27B-Opus-Distill-v2 — bugfix release, see Known Issues above.

Qwen3.8-27B-Opus-Distill

README

Known issues

Model overview

Highlights

Benchmark results

Important caveats

Quick start

Related models

Explore FriendliAI today

README

Known issues

Model overview

Highlights

Benchmark results

Important caveats

Quick start

Related models