matboz

qwen3.6-27b-difficult-advice-tulu-lora

README

License: apache-2.0

Training mixture

Table with columns: Source, Examples, Tokens, Share, Rendering
Source	Examples	Tokens	Share	Rendering
`matboz/difficult-advice-qwen3` (`sft_dataset_thinking.jsonl`)	291	299,455	20.0%	with `<think>` reasoning traces
`allenai/tulu-3-sft-mixture`	1,878	1,194,548	80.0%	no `<think>` block at all
Total	2,169	1,489,959

Why the TULU3 replay data carries no think block

Qwen3.6's chat template renders <think>\n{reasoning}\n</think> for any assistant turn that is the final message. Trace-free replay data would therefore render with an empty <think></think>, which trains the model to stop reasoning — the documented failure mode that motivated the thinking format in the first place. The mixture builder instead appends a throwaway user turn so the template takes its no-think branch, then strips that turn. The built dataset is asserted to contain zero empty think blocks, with think blocks in exactly the 291 difficult-advice rows.

Training

bf16 LoRA (not QLoRA — bitsandbytes 4-bit does not reliably cover this model's hybrid linear-attention/SSM layers), 1×H100 80GB.

Table

r / alpha / dropout	32 / 64 / 0.05
target modules	regex scoped to `model.language_model.*` (q/k/v/o/gate/up/down proj)
epochs	1
batch × grad-accum	1 × 16 (136 optimizer steps)
lr / schedule	1e-4, cosine, 3% warmup, annealed to 0
max seq len / packing	2048 / off
runtime	1h38m

The regex is scoped to model.language_model so the vision tower (model.visual) is left untouched. Packing is disabled because TRL only guarantees packed-sequence isolation under Flash Attention variants; under sdpa it warns of cross-contamination between samples.

Loss: 2.93 → ~1.00 by step 15, then flat (0.89–1.13). Final token accuracy 0.728, grad_norm 0.31.

Caveats

1 epoch — half the gradient steps of the Qwen3-32B run this replicates. If a downstream eval shows no effect, undertraining is a live alternative explanation.
The difficult-advice signal is only 20% of tokens, and most of the loss drop is early format adaptation, so its influence on the final weights is comparatively weak.
Not yet evaluated on ODCV-Bench at time of upload.

Usage

python
from peft import PeftModel
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "matboz/qwen3.6-27b-difficult-advice-tulu-lora")
model = model.merge_and_unload()  # vLLM LoRA support for this hybrid arch is unproven

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

matboz

Model Tree

Base

Qwen/Qwen3.6-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality