GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
LASR-Callum
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
LASR-Callum
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
Published in full as
LASR-Callum/2026-07-31-toolcalling-tulu-20-80-mixture.
| Source | Examples | Tokens | Share | Rendering |
|---|---|---|---|---|
agentic tool-use (approved_agentic, fullthink) | 124 | 297,894 | 19.96% | reasoning kept where the source had it; no empty think blocks |
| TULU3 replay | 1,878 | 1,194,548 | 80.04% | no <think> block at all |
| Total | 2,002 | 1,492,442 |
25 of the 124 agentic documents actually emit tool calls
— 92 <tool_call> spans in Qwen3.6's XML dialect, all verified balanced.
bf16 LoRA (not QLoRA — bitsandbytes does not reliably cover this model's hybrid linear-attention/SSM layers), 1×H100 80GB SXM, 1h38m09s.
| r / alpha / dropout | 32 / 64 / 0.05 |
| target modules | regex scoped to model.language_model.* (q/k/v/o/gate/up/down proj) |
| epochs / steps | 1 / 126 |
| batch x grad-accum | 1 x 16 |
| lr / schedule | 1e-4, cosine, 3% warmup, annealed to 0 |
| max seq len / packing | 4096 / off |
assistant_only_loss | false |
| seed |
Loss: 2.7528 → 1.057 (epoch average); the last logged step (125) read 1.0199. Epoch-average mean token accuracy 0.7071, final grad_norm 0.3573, 1,492,498 tokens consumed.
Curves and the full log history are in
LASR-Callum/2026-07-31-toolcalling-tulu-sft-run.
max_seq_len is 4096 and the sibling arms use 2048These agentic conversations run 9–13 turns with a median of 2,348 tokens, and 99 of the 151 source
documents exceed 2048. Measured: a 2048 cap keeps only 80.4% of the corpus and severs 11 of its
98 <tool_call> spans, inside exactly the long conversations the tool calls live in. At 4096 the
mixture is truncated nowhere at all. The cost is that this arm differs from its siblings on one
hyperparameter as well as on composition — read the head-to-head with that caveat.
target_modules only half-applies. Qwen3.6-27B is hybrid: q/k/v/o_proj exist in 16 of 64
layers, the rest being linear-attention blocks with different module names. gate/up/down_proj
attach to all 64. So this adapter tunes MLP throughout but attention in only a quarter of the
stack. Same for every arm in the family, so comparisons are unaffected.python
from peft import PeftModelfrom transformers import AutoModelForImageTextToTextmodel = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")model = PeftModel.from_pretrained(model, "LASR-Callum/qwen3.6-27b-toolcalling-tulu-lora-20-80")model = model.merge_and_unload() # vLLM LoRA support for this hybrid arch is unproven
Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language
checkpoint. Merging drops the base model's 15 mtp.* tensors, so speculative decoding needs them
grafted back.
Repository https://github.com/Matthew-Bozoukov/teaching_claude_why_replication @ 639d85c.
Built with src/experiments/build_toolcalling_mixture.py, trained with
src/experiments/train_lora.py --config configs/train_lora_toolcalling.yaml.
| 0 |
| trainable params | 159.4M |