qwen3.6-27b-reasoning-distill-fp8-v1 API & Inference Endpoint

Training data (1,652 examples; 1,247 effective after a 16,384-token cap)

50 percent reasoning examples (826): multi-turn, tool-calling, vision-grounded agentic traces with explicit thinking, harvested from an interactive visual-reasoning environment. Median ~14.7k tokens; completion-only loss on the assistant turn (open think block + tool call).
50 percent distillation corpus (826): sampled (seed 0) from r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation sft_balanced — teachers ~86 percent qwen3.8-max-preview, ~10 percent GLM-5.2, ~4 percent Kimi K3; domains reasoning / agent-tool / math / code / science; verifier-failed rows dropped.
Validation: 73 reasoning examples held out by environment (no overlap with train).

Recipe & numbers

LoRA r=32, alpha=64, dropout 0 — 304 target modules: q/k/v/o_proj on 16 full-attention layers plus all five unpacked linear-attention projections (in_proj_qkv/z/a/b, out_proj) on 48 gated-DeltaNet layers. 95,043,584 trainable params (0.346 percent). Vision tower and lm_head frozen.
QLoRA NF4 (double-quant, bf16 compute), completion-only masking rendered with the model's real chat template (prefix-exact token masking, 466/466 verified).
2 epochs = 312 steps, batch 1 x grad-accum 8, lr 1e-4 cosine, A100-80GB ~14 h.
Loss: epoch 1 train 0.325 / val 0.249 - epoch 2 train 0.220 / val 0.254.
Held-out generation check (n=16): 14/16 parse+compile both arms — zero regressions.

Export chain

BF16 merge (top-1 logit parity gate vs the pre-merge reference passed) → llm-compressor FP8 static per-tensor W8A8, calibrated on 96 on-distribution prompts. Linear-attention sublayers, vision tower and lm_head stay BF16 (see recipe.yaml).

Serving

markdown
vllm serve <this-repo> --max-model-len 32768 \
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder

Thinking on: chat_template_kwargs={"enable_thinking": true}, temp 0.6 / top_p 0.95 / top_k 20.

Limitations

v1 trained text-only (images placeholder-tokenized at train time; served multimodal). Small eval n; the adapter targets agentic tool-use reasoning and is not a general-purpose improvement claim.

Sibling repo: the unmerged adapter is published as Ravionhf/qwen3.6-27b-reasoning-distill-lora-v1.

Training data (1,652 examples; 1,247 effective after a 16,384-token cap)

50 percent reasoning examples (826): multi-turn, tool-calling, vision-grounded agentic traces with explicit thinking, harvested from an interactive visual-reasoning environment. Median ~14.7k tokens; completion-only loss on the assistant turn (open think block + tool call).
50 percent distillation corpus (826): sampled (seed 0) from r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation sft_balanced — teachers ~86 percent qwen3.8-max-preview, ~10 percent GLM-5.2, ~4 percent Kimi K3; domains reasoning / agent-tool / math / code / science; verifier-failed rows dropped.
Validation: 73 reasoning examples held out by environment (no overlap with train).

Recipe & numbers

LoRA r=32, alpha=64, dropout 0 — 304 target modules: q/k/v/o_proj on 16 full-attention layers plus all five unpacked linear-attention projections (in_proj_qkv/z/a/b, out_proj) on 48 gated-DeltaNet layers. 95,043,584 trainable params (0.346 percent). Vision tower and lm_head frozen.
QLoRA NF4 (double-quant, bf16 compute), completion-only masking rendered with the model's real chat template (prefix-exact token masking, 466/466 verified).
2 epochs = 312 steps, batch 1 x grad-accum 8, lr 1e-4 cosine, A100-80GB ~14 h.
Loss: epoch 1 train 0.325 / val 0.249 - epoch 2 train 0.220 / val 0.254.
Held-out generation check (n=16): 14/16 parse+compile both arms — zero regressions.

Export chain

Serving

markdown
vllm serve <this-repo> --max-model-len 32768 \
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder

Thinking on: chat_template_kwargs={"enable_thinking": true}, temp 0.6 / top_p 0.95 / top_k 20.

Limitations

v1 trained text-only (images placeholder-tokenized at train time; served multimodal). Small eval n; the adapter targets agentic tool-use reasoning and is not a general-purpose improvement claim.

Sibling repo: the unmerged adapter is published as Ravionhf/qwen3.6-27b-reasoning-distill-lora-v1.

qwen3.6-27b-reasoning-distill-fp8-v1

README

Training data (1,652 examples; 1,247 effective after a 16,384-token cap)

Recipe & numbers

Export chain

Serving

Limitations

Explore FriendliAI today

README

Training data (1,652 examples; 1,247 effective after a 16,384-token cap)

Recipe & numbers

Export chain

Serving

Limitations