GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
LASR-Callum
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
LASR-Callum
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
The TULU3 (119) and No Robots (271) rows each carry <think>\n\n</think>\n\n at the
start of their final assistant turn -- Qwen3.6's non-thinking marker, placed exactly where
apply_chat_template puts it (asserted against the template before any data was touched).
The 611 NuminaMath-CoT rows carry no marker: their chain-of-thought lives in the response
text, so marking them "non-thinking" would contradict their own content.
Those marker tokens are excluded from the loss. The model is conditioned on the marker -- which is how Qwen3.6 injects it as a prefill in non-thinking mode -- but is never trained to emit one, since learning to emit an empty think block is the documented reasoning-collapse pattern.
markdown
<|im_start|> MASKEDassistant MASKED<think> MASKED <- marker: context, not a target</think> MASKEDA LOSS <- supervision starts at the real answer
The arithmetic confirms it: 390,921 supervised tokens, identical to the marker-free build of the same mixture. Adding 390 markers x 4 tokens changed the supervised count by zero.
| Supervised | 390,921 / 499,595 = 78.2% |
| Epochs / steps | 1 / 63 |
| Runtime | 33 min, 1x H100 80GB |
| r / alpha / dropout | 32 / 64 / 0.05 |
| batch x grad-accum | 1 x 16 |
| lr / schedule | 4e-5, cosine, 3% warmup |
| max seq len / packing | 3072 / off |
| Final loss | 0.878 |
| Token accuracy | 0.793 |
loss_type: nll is set because TRL's default chunked-CE path patches the LM head and reads
forward.__func__, which fails on this checkpoint's functools.partial forward. The loss is
mathematically the same.
| Run | lr | Markers | Loss | Token acc |
|---|---|---|---|---|
| 500k maths-weighted | 1e-4 | none | 0.897 | 0.798 |
| this | 4e-5 | present, masked | 0.878 | 0.793 |
Two variables differ between these (learning rate and the markers), so the pair does not isolate either. A marker-free run at 4e-5 would be needed for that.
python
from peft import PeftModelfrom transformers import AutoModelForImageTextToTextmodel = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")model = PeftModel.from_pretrained(model, "LASR-Callum/qwen3.6-27b-lora-500k-numina-heavy-empty-think")model = model.merge_and_unload()
Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language
checkpoint.