Training details
- Method: LoRA (peft), merged into the base weights in this repo
- LoRA: r=64, alpha=128, dropout=0.05
- Targets (310 modules): attention
q/k/v/o_proj, linear-attention in_proj_a/b/qkv/z + out_proj,
shared-expert gate/up/down_proj (routed experts untouched — 85% of weights)
- Data: 14.4k examples stratified:
- Reasoning: 6k (OpenR1-Math + simplescaling/s1K, 5 effort levels × ~1200 each)
- Code (local corpus): 4.9k (Rust 1101, QML 900, JS 900, Py 900, TS 711, CSS 250, HTML 66, Bash 100)
- Code (instruction): 800 (Magicoder-Evol-Instruct-110K, capped per language)
- Tool use: 1.5k (NousResearch Hermes + glaive)
- Bilingual chat: 1.2k (OpenAssistant oasst2, 21% RU)
- Loss target: ~1.4 to avoid memorization; stopped at step 1500 of 2436 (train loss 0.43)
- Effort data: reasoning traces regenerated at 5 budget levels so the marker maps to real behaviour
- Hardware: single A100-80GB, 4-bit base + bf16 LoRA, ~4 h
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto") msgs = [ {"role": "system", "content": "You are a coding and reasoning assistant.\nreasoning_effort=high\nAdjust the depth of your reasoning to the level given above."}, {"role": "user", "content": "Напиши функцию бинарного поиска на Python."},]enc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)out = model.generate(enc["input_ids"].to(model.device), max_new_tokens=800)print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
Reasoning effort control
The effort level is read from the system prompt:
<system>You are a coding and reasoning assistant.reasoning_effort=mediumAdjust the depth of your reasoning to the level given above.</system>
Table with columns: Value, Behaviour| Value | Behaviour |
|---|
low | 1–2 sentences, minimal reasoning |
medium | short paragraph |
high | multi-paragraph reasoning |
xhigh | long trace, explores alternatives |
max | full chain-of-thought |
This was trained with matched-pairs data (same question at 5 effort levels), so the mapping is
behavioural rather than a truncation of the same text.
Known eval results (baseline → finetuned)
Table with columns: Metric, Baseline, Finetuned| Metric | Baseline | Finetuned |
|---|
| Tool Use (8 scenarios) | 6/8 | 7/8 |
| Agentic (5 tasks) | 5/5 | 5/5 |
| Effort monotonic (low→max) | ❌ | ✅ |
| RU Perplexity | — | 5.41 |
Full benchmark suite (HumanEval, MBPP, GSM8K, MMLU, SWE-bench) is planned.
Licence
Apache-2.0, inherited from Qwen/Qwen3.6-35B-A3B.