LASR-Callum

qwen3.6-27b-difficult-advice-tulu-lora-20-80

README

License: apache-2.0

Mixture-ratio sweep

This is the 20/80 arm and the only one evaluated so far. All three hold total tokens at ~1.493M with identical hyperparameters (1 epoch, packing off), so the mixture ratio is the only variable.

Table with columns: Arm, Difficult-advice, TULU3, ODCV-Bench
Arm	Difficult-advice	TULU3	ODCV-Bench
10/90	149,816 tok	1,343,608 tok	not yet run
20/80 (this)	299,455 tok	1,194,548 tok	37.2% → 19.2%
40/60	597,013 tok	896,346 tok	not yet run

Raw transcripts behind the 20/80 result: matboz/odcv-qwen3.6-27b-transcripts.

Training mixture

Table with columns: Source, Examples, Tokens, Share, Rendering
Source	Examples	Tokens	Share	Rendering
`matboz/difficult-advice-qwen3` (`sft_dataset_thinking.jsonl`)	291	299,455	20.0%	with `<think>` reasoning traces
`allenai/tulu-3-sft-mixture`

Why the TULU3 replay data carries no think block

Qwen3.6's chat template renders <think>\n{reasoning}\n</think> for any assistant turn that is the final message. Trace-free replay data would therefore render with an empty <think></think>, which trains the model to stop reasoning — the documented failure mode that motivated the thinking format in the first place. The mixture builder instead appends a throwaway user turn so the template takes its no-think branch, then strips that turn. The built dataset is asserted to contain zero empty think blocks, with think blocks in exactly the 291 difficult-advice rows.

Training

bf16 LoRA (not QLoRA — bitsandbytes 4-bit does not reliably cover this model's hybrid linear-attention/SSM layers), 1×H100 80GB.

Table

r / alpha / dropout	32 / 64 / 0.05
target modules	regex scoped to `model.language_model.*` (q/k/v/o/gate/up/down proj)
epochs	1
batch × grad-accum	1 × 16 (136 optimizer steps)
lr / schedule	1e-4, cosine, 3% warmup, annealed to 0
max seq len / packing	2048 / off
runtime	1h38m

The regex is scoped to model.language_model so the vision tower (model.visual) is left untouched. Packing is disabled because TRL only guarantees packed-sequence isolation under Flash Attention variants; under sdpa it warns of cross-contamination between samples.

Loss: 2.93 → ~1.00 by step 15, then flat (0.89–1.13). Final token accuracy 0.728, grad_norm 0.31.

Result on ODCV-Bench

Evaluated against the unmodified base model served identically (same vLLM 0.26 build, same --quantization fp8, same flags, temperature 0, same 78 scenario cells, same two judges — Grok-4.20 + Gemini-3.1-Pro, median-of-2). The arms differ only by this adapter.

Table with columns: Metric, Base Qwen3.6-27B (FP8), + this LoRA, Paired difference
Metric	Base Qwen3.6-27B (FP8)	+ this LoRA	Paired difference
Misalignment rate	37.2%	19.2%	−17.9 pp, 95% CI [−26.9, −9.0]
Mean severity	1.43	0.87	−0.56, 95% CI [−0.88, −0.26]

A 48% relative reduction. Per-scenario, 15 cells flipped from base-violation to safe and 1 went the other way (McNemar exact two-sided p = 0.00052), across healthcare, finance, audit, hiring, propaganda and legal domains in both framings.

The base arm scores 37.2% against the paper's published 43.8% for this model, with the published value inside the base arm's 95% CI — so the control reproduces published behaviour and the drop is not a serving artifact.

Caveats

1 epoch — half the gradient steps of the Qwen3-32B run this replicates, with difficult-advice at only 20% of tokens. The effect appears despite a light dose, not because of a heavy one; it is untested whether more training increases or degrades it.
Median-of-2 judges, not the paper's median-of-4, so the absolute numbers are not directly comparable to the published 43.8%. The internal base-vs-LoRA comparison is unaffected since both arms use the same two judges.
Two scenario cells were excluded from both arms (they ran >20 min without terminating). Both are known-violation cells for the base, so absolute MR is lowered equally for both arms.
n=1 trajectory per cell at temperature 0, as in the benchmark's own protocol.
Unpaired confidence intervals overlap; the result rests on the paired analysis, which is valid here because both arms ran an identical scenario set.

Usage

python
from peft import PeftModel
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "LASR-Callum/qwen3.6-27b-difficult-advice-tulu-lora-20-80")
model = model.merge_and_unload()  # vLLM LoRA support for this hybrid arch is unproven

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

LASR-Callum

Model Tree

Base

Qwen/Qwen3.6-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality