Results (v2, trained on 360 edit pairs)
Held-out eval (8 unseen base structures), and a blind A/B quality judge vs Sonnet's
edit of the same (original, instruction):
Table with columns: Metric, v1 (240 pairs), v2 (360 pairs)| Metric | v1 (240 pairs) | v2 (360 pairs) |
|---|
| Produces valid renderable .arr | 8/8 | 8/8 |
| Actually applies the requested edit | 7/8 | 8/8 |
| Blind quality vs Sonnet | 2/8 | 1 win + 2 byte-identical ties + 5 losses |
On 2 held-out specs (brass-stab, arpeggio-conversion) v2 produced edits byte-for-byte
identical to Sonnet's — it learned those idioms exactly. The remaining gap to Sonnet
is pitch-level precision: the losses are "close but imprecise" (a reharmonization
with the right chord concept but pitches transposed +2 semitones; a countermelody with
2 out-of-key notes; an overshot climax). This mirrors the ~4B capacity ceiling seen on
the from-scratch compose task — but editing is more tractable because the original
arrangement supplies in-key scaffolding.
Recommended use: hybrid — gemma for fast structural edits (instrument swaps, voice
adds, arpeggiation, drum-feel changes), Sonnet for precision-critical edits
(reharmonization, exact voice-leading).
Files
adapter_model.safetensors, adapter_config.json — the LoRA
tokenizer*.json, chat_template.jinja — tokenizer + chat template
edit_training_pairs.jsonl — 360 (original, instruction, edited) pairs (Sonnet-generated), trained x12 key transpositions
blind_judge_verdicts_v2.json — per-spec blind-judge outcomes
edit_specs.py, build_edit_train.py, eval_edit.py, edit_render_check.py — reproducible pipeline
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("google/gemma-4-E4B-it", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "hidude561/justice")
tok = AutoTokenizer.from_pretrained("hidude561/justice")
Trained with LoRA (bf16), 3 epochs, max_len 3072, on an RTX 5090.