GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
LASR-Callum
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
LASR-Callum
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
Every TULU3 replay row carries <think>\n\n</think>\n\n on its final assistant turn --
Qwen3.6's non-thinking marker, placed exactly where apply_chat_template puts it (the template
emits it only on final turns; the insertion is asserted to reproduce the template byte-for-byte
before any data is touched).
Those marker tokens are masked out of the loss. The model is conditioned on the marker --
which is how Qwen3.6 injects it as a prefill in non-thinking mode -- but never trained to emit
it, since learning to emit an empty think block is the documented reasoning-collapse pattern.
Difficult-advice rows are untouched and keep their real <think> traces fully supervised.
markdown
<|im_start|> MASKEDassistant MASKED<think> MASKED <- marker: context, not a target</think> MASKEDTo LOSS <- supervision starts at the answer
| Source | Rows | Tokens | Marker | Supervised |
|---|---|---|---|---|
| difficult-advice | 147 | 149,816 | 0 | 85.55% |
| TULU3 replay | 2,110 | 1,352,044 | 2,110 | 77.36% |
| Total | 2,257 | 1,501,860 | 2,110 | 78.18% |
Supervision is otherwise assistant-tokens-only: everything outside an assistant turn is -100.
A supervised span ends after the closing <|im_end|>, which the model must produce to stop.
bf16 LoRA (not QLoRA — bitsandbytes does not reliably cover this model's hybrid linear-attention/SSM layers: 48 of 64 layers are Gated DeltaNet and none of their projections receive an adapter). 1×H100 80GB, 96 min.
| r / alpha / dropout | 32 / 64 / 0.05 |
| target modules | regex scoped to model.language_model.* (q/k/v/o/gate/up/down proj) |
| epochs / steps | 1 / 142 |
| batch × grad-accum | 1 × 16 |
| lr / schedule | 1e-4, cosine, 3% warmup |
| max seq len / packing | 2048 / off |
Final train loss 0.816, token accuracy 0.752.
Not yet evaluated on ODCV-Bench or agentic-misalignment. For reference, the full-token sweep at the same budget:
| Difficult-advice share | ODCV-Bench MR | Agentic-misalignment |
|---|---|---|
| 0% (base) | 37.2% | 65.5% |
| 10% | 24.7% | 38.7% |
| 20% | 19.2% | 25.3% |
| 40% | 15.4% | 19.5% |
On the 20/80 arm, adding the marker cut median reasoning-trace length from 700 to 234 tokens (base 1153) with no collapse — zero traces under 3 tokens. The effect was concentrated on everyday-advice prompts, i.e. it teaches when reasoning is unnecessary rather than suppressing it.
Sibling empty-think arms: 10_90 · 80_20 · 40_60
python
from peft import PeftModelfrom transformers import AutoModelForImageTextToTextmodel = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")model = PeftModel.from_pretrained(model, "LASR-Callum/2026-08-01-qwen36-difficult-advice-tulu-lora-10-90-empty-think-tags")model = model.merge_and_unload()
Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language
checkpoint. Merging drops the base model's 15 mtp.* tensors.