basta

lightning-mcqwen3

README

License: apache-2.0

Files

adapter_config.json, adapter_model.safetensors: the LoRA (r16, alpha 32, all linear layers), trained from Qwen3-0.6B at revision c1899de289a04d12100db370d81485cdf75e47ca.
gate.pt: an optional think gate for tool calls. A logistic probe on the adapted model's layer-21 hidden state at the end of the prompt; where it fires, the reply starts with an empty thinking block. It only applies to prompts with a function list and is used through the repository's evaluate.py --gate.

Results

Against Qwen3-0.6B, 8 samples per question, 95% intervals from a paired bootstrap over questions:

Table with columns: Tokens saved, Accuracy change
	Tokens saved	Accuracy change
SVAMP (maths)	66.3%	−3.8 pp
MBPP+ (code)	39.1%	−1.9 pp
IFEval (instructions)	32.0%	+1.4 pp
BFCL (tool calls)	44.6%	+0.8 pp
All	47.3% [45.8, 48.7]	−0.88 pp [−1.61, −0.15]

Without the gate, the adapter alone saves 43.2% at −0.85 pp; the gate only changes BFCL (25.3% → 44.6% saved there). The headline combines the ungated runs on SVAMP, MBPP+ and IFEval, which the gate doesn't touch, with a gated BFCL run. A full gated rerun read 46.6% / −1.33 pp, within rerun noise.

Use

With transformers and peft, without the gate:

python

from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", torch_dtype="auto")model = PeftModel.from_pretrained(model, "basta/lightning-mcqwen3") messages = [{"role": "user", "content": "A train travels 60 km in 45 minutes. What is its speed in km/h?"}]ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")out = model.generate(ids, max_new_tokens=1024, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

The evaluation samples with temperature 0.6, top-p 0.95, top-k 20, Qwen3's recommended thinking-mode settings. To score it on the benchmarks, with the gate:

bash

git clone https://github.com/basta/lightning-mcqwen3 && cd lightning-mcqwen3hf download basta/lightning-mcqwen3 --local-dir weightspython evaluate.py --out runs/final --adapter weights --gate weights/gate.pt

Training

Every step is a LoRA continued from the one before, on the challenge's suggested training data only:

Prompt distillation. SFT on Chain-of-Draft traces with the prompt removed (11,247 targets).
On-policy oracle DPO. The distilled model's correct traces are cut at 10–90% of their thinking and re-answered 4 times from each cut. DPO prefers the earliest cut from which all 4 re-answers stay right (9,034 pairs, stopped at step 100 of 561).
Anchored correctness DPO. The model's own right maths trace over its own wrong one at matched length, with an NLL term on the right one (732 pairs, 42 steps).
Think gate. Fitted on 1,500 ToolACE prompts, labelled by whether thinking changes the sampled tool call; no correctness labels and no benchmark data.

The repository's scripts/reproduce.sh runs every step with its settings.

Credit

The challenge, evaluation harness and training data selection are by BottleCapAI. The prompt is Chain of Draft (Xu et al., 2025). The truncation oracle with guard pairs is close to ThoughtFold (ICML 2026), and the probe gate is a label-free variant of ThinkSwitcher (EMNLP Findings 2025).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

basta

Model Tree

Base

Qwen/Qwen3-0.6B

Adapter

this model

Input Modalities

Text

Output Modalities