Benchmark Results
The MMLU-Pro pass used 70 total questions per model: --limit 5 across 14 MMLU-Pro subjects. Treat this as a smoke/comparative check, not a release-quality full benchmark.
Table with columns: Benchmark, Harness, Samples per model, Setting, Metric, Base model, Fine-tuned merged model, Delta| Benchmark | Harness | Samples per model | Setting | Metric | Base model | Fine-tuned merged model | Delta |
|---|
| MMLU-Pro overall | lm-evaluation-harness | 70 | --limit 5 across 14 subjects | exact_match, custom-extract | 42.86% | 75.71% | +32.85 pp |
Base model: Qwen/Qwen3.6-35B-A3B. Fine-tuned model: hesamation/Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled.
[!WARNING]
Community benchmarks welcome
To better understand this fine-tuned model's capabilities, I welcome independent benchmark results. If you run evaluations, please include the benchmark name, harness/script, sample count, decoding settings, and raw logs or result files when possible.
Share results by opening a PR/discussion or DMing @hesamation on X.
Base Qwen3.6 Highlights
This release delivers substantial upgrades, particularly in:
- Agentic Coding: the model now handles frontend workflows and repository-level reasoning with greater fluency and precision.
- Thinking Preservation: Qwen introduced a new option to retain reasoning context from historical messages, streamlining iterative development and reducing overhead.
For more details, please refer to the Qwen blog post Qwen3.6-35B-A3B.
Base Model Overview
- Type: Causal Language Model with Vision Encoder
- Training Stage: Pre-training & Post-training
- Language Model:
- Number of Parameters: 35B in total and 3B activated
- Hidden Dimension: 2048
- Token Embedding: 248320 (Padded)
- Number of Layers: 40
- Hidden Layout: 10 x (3 x (Gated DeltaNet -> MoE) -> 1 x (Gated Attention -> MoE))
- Gated DeltaNet:
- Number of Linear Attention Heads: 32 for V and 16 for QK
- Head Dimension: 128
- Gated Attention:
- Number of Attention Heads: 16 for Q and 2 for KV
- Head Dimension: 256
- Rotary Position Embedding Dimension: 64
Base Benchmark Results
The following table is from the upstream Qwen3.6-35B-A3B release context and is included for base-model reference. It is not a benchmark of this fine-tuned checkpoint unless explicitly stated in the fine-tune benchmark table above.
Table with columns: Category, Benchmark, Qwen3.5-27B, Gemma4-31B, Qwen3.5-35BA3B, Gemma4-26BA4B, Qwen3.6-35BA3B| Category | Benchmark | Qwen3.5-27B | Gemma4-31B | Qwen3.5-35BA3B | Gemma4-26BA4B | Qwen3.6-35BA3B |
|---|
| Coding Agent | SWE-bench Verified | 75.0 | 52.0 | 70.0 | 17.4 | 73.4 |
| Coding Agent | SWE-bench Multilingual | 69.3 | 51.7 | 60.3 | 17.3 |
Notes from the upstream Qwen3.6 release:
- SWE-Bench Series: internal agent scaffold with bash and file-edit tools; temp=1.0, top_p=0.95, 200K context window.
- Terminal-Bench 2.0: Harbor/Terminus-2 harness; 3h timeout, 32 CPU/48 GB RAM; temp=1.0, top_p=0.95, top_k=20, max_tokens=80K, 256K ctx; average of 5 runs.
- SkillsBench: evaluated via OpenCode on 78 tasks, using a self-contained subset excluding API-dependent tasks; average of 5 runs.
- NL2Repo: evaluated via Claude Code for other models, with temp=1.0, top_p=0.95, max_turns=900.
- QwenClawBench: internal real-user-distribution Claw agent benchmark; temp=0.6, 256K ctx.
- QwenWebBench: internal front-end code generation benchmark; bilingual EN/CN, seven categories, auto-render plus multimodal judge, BT/Elo rating system.
- TAU3-Bench: official user model with gpt-5.2 low reasoning effort and default BM25 retrieval.
- VITA-Bench: average subdomain scores, using claude-4-sonnet as judge.
- MCPMark: GitHub MCP v0.30.3, Playwright responses truncated at 32K tokens.
- MCP-Atlas: public set score, gemini-2.5-pro judge.
- AIME 26: full AIME 2026 I and II.
Training Pipeline
Qwen/Qwen3.6-35B-A3B
-> supervised fine-tuning with LoRA
-> merged full model
-> Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled
Training configuration:
Table with columns: Setting, Value| Setting | Value |
|---|
| Fine-tuning method | Supervised fine-tuning with LoRA |
| LoRA target | Attention-only modules |
| LoRA rank / alpha | 32 / 32 |
| Micro-batch size | 1 |
| Gradient accumulation | 32 |
| Epochs | 2 |
| Completed steps | 762 / 762 |
| Final reported training loss | 0.3362497625740494 |
| Dataset max tokens |
Training Data
The recipe samples and normalizes reasoning conversations from three datasets, then renders them with the qwen3-thinking chat template and response-only SFT masking.
Intended Use
This model is intended for reasoning-heavy text workflows such as coding assistance, planning, math-style reasoning, and structured analytical responses. Because the fine-tune is text-only, image/video behavior should be treated as inherited from the base model rather than improved by this training run.
Acknowledgements
Thanks to the Qwen team for the base model, Unsloth for the training stack, and Jackrong for the public reasoning-distillation workflow that inspired this fine-tune.