LASR-Callum

qwen3.6-27b-lora-t2-9284-da-chunk-only-702-r64-dynbatch

README

License: apache-2.0

What was ablated

The baseline difficult-advice recipe injects the WHOLE constitution into exactly two of its five LLM stages — revise_prompts and revise_responses. Every other stage already sees only its one target chunk. This arm's corpus deleted those two injections, so no stage ever saw more than one principle at a time.

That also withholds the constitution's preamble — the priority / conflict-resolution section saying how principles trade off. It belongs to no chunk, so at principle granularity it reached the generator only through the removed slots.

A measured side effect of the ablation

Generating the ablated corpus, Anthropic's content filter refused revise_prompts calls at roughly 6% against the baseline's ~0% on the identical scenarios, and the refusals were deterministic across a resample. Without the constitution framing the request as alignment training data, the prompt reads as "make this manipulative message a sharper test", and both the provider filter and the model's own output-contract compliance degrade. The corpus finished at 708 rows of an intended 716.

Table with columns: field, value
field	value
`base_model`	`Qwen/Qwen3.6-27B`
`thinking`	`True` — trained on real reasoning traces; evaluate in thinking mode
`training_data`	`LASR-Callum/2026-08-21-table2-9284-da-chunk-only-702-train` :: `t2_9284_da_chunk_only_702.jsonl` @ `27e8f532187e`
`corpus`	`LASR-Callum/2026-08-21-difficult-advice-v2-chunk-only-716`
`constitution`	`constitutions/claude_distilled_12_principles_mid/constitution.md` — generated FROM it, refine stages NOT shown it
`train_config`	`configs/train/lora_qwen36_t2_9284_da_chunk_only_702_dynbatch_2xh200.yaml`
`source_repo`	https://github.com/Matthew-Bozoukov/Lessons_from_constituitional_AFT.git @ `4eaa969be9c82433dc28b05bf78f09922179ce51`
`hardware`	2xH200, DDP, token-budgeted dynamic batching (8,000 padded tokens/GPU)
`mixture`	9,986 rows = 702 difficult-advice-chunk-only + 9,284 Table-2 (7.03% DA)
`steps`	625 (1 epoch, global batch 16), final logged loss 0.7758
`lora`	r=64, alpha=128, dropout=0.05

Mask gate verified on-pod before training: 64 rows decode-verified, think census 702 real / 9,646 empty (93.2% of turns) / 0 absent.

Provenance

markdown
uv run synth run --config configs/data/synth/difficult_advice_chunk_only.yaml \
  --resume output/synthdoc_v2_chunk_only/20260821_paired716
uv run python scratch/build_t2_9284_da716_mixture.py \
  --synth_repo LASR-Callum/2026-08-21-difficult-advice-v2-chunk-only-716 \
  --synth_file dataset.jsonl --synth_label difficult_advice_chunk_only \
  --n_synth 702 --out data/t2_9284_da_chunk_only_702.jsonl --seed 0
torchrun --nproc_per_node=2 scripts/train/train_lora.py \
  --config configs/train/lora_qwen36_t2_9284_da_chunk_only_702_dynbatch_2xh200.yaml

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

LASR-Callum

Model Tree

Base

Qwen/Qwen3.6-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality