suunny

music-response-distillation

README

License: apache-2.0

What is included

This repository contains the learned PEFT LoRA weights, adapter configuration, and checkpoint tokenizer/chat template. It requires the separate Qwen/Qwen3.8-27B base model; it is not a merged standalone 27B model. The adapter is saved as safetensors. Optimizer state, private logs, credentials, and teacher sessions are excluded.

The base revision used by the experiment is 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. File checksums and checkpoint provenance are in manifest.json.

Load the adapter

Use a CUDA environment with enough GPU memory for the 27B base and generation cache. The example loads the base in 4-bit NF4 and attaches the adapter using PEFT.

pip install "transformers==5.18.0" "peft==0.20.0" accelerate bitsandbytes torch

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel adapter_id = "suunny/music-response-distillation"base_id = "Qwen/Qwen3.8-27B"base_revision = "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0" tokenizer = AutoTokenizer.from_pretrained(adapter_id)base = AutoModelForCausalLM.from_pretrained(    base_id,    revision=base_revision,    quantization_config=BitsAndBytesConfig(        load_in_4bit=True,        bnb_4bit_quant_type="nf4",        bnb_4bit_use_double_quant=True,        bnb_4bit_compute_dtype=torch.bfloat16,    ),    dtype=torch.bfloat16,    device_map="auto",)model = PeftModel.from_pretrained(base, adapter_id).eval()messages = [{"role": "user", "content": (    "Write an 8-bar melody in C major, 4/4 time, for one piano voice. "    "Return ABC notation.")}]inputs = tokenizer.apply_chat_template(    messages,    tokenize=True,    add_generation_prompt=True,    enable_thinking=False,    return_dict=True,    return_tensors="pt",).to(model.device)with torch.inference_mode():    output = model.generate(**inputs, max_new_tokens=2048, do_sample=False)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

This is a usage example, not a reproduction of the reported evaluation. The published results used vLLM with FP8 base weights, an 8,192-token output limit, and the project's structural generation guard. Quantization, output budget, and stopping rules can change outputs and scores. The snippet has not been run as a new GPU inference job for this release. For the exact evaluation path, see the run instructions.

Training flow

mermaid

flowchart TD    A[1,000 source music prompts] --> B[806 training prompts]    A --> C[105 validation + 89 test prompts]    B --> D[Teacher: gpt-6.1-sol / low reasoning effort]    D --> E[ABC syntax, rhythm, voices, bar counts and reward checks]    E --> F[607 accepted responses]    E --> G[199 rejected responses / analysis only]    F --> H[Completion-only QLoRA SFT / one H100]    H --> I[Epoch checkpoints: 38, 76, 114]    C --> J[Base + checkpoint evaluation / separate H100 worker]    I --> J    J --> K[Verifier scores, ABC outputs and playback artifacts]

Prepare prompts. Deterministically split the source music tasks into training, validation, and test partitions.
Collect and verify. Generate one final teacher answer per training prompt. Retain accepted answers for SFT and preserve rejected answers with their reasons.
Train the student. Predict the next token of the teacher's answer, masking prompt and padding tokens from the loss. Update LoRA parameters with cross-entropy.
Evaluate checkpoints. Compare the unadapted base model and each epoch using fresh generations. Training-set and held-out scores have separate denominators; aborted generations remain zero-reward outcomes.

The learning method is off-policy response distillation: teacher text supplies hard token targets for SFT. Teacher logits are not collected, and this run does not apply a reinforcement-learning update. The results below cover validation; publishing test prompts does not establish a reported test-set result.

Results

All four checkpoints use the same 105 validation prompts.

Table with columns: Metric, Base, Epoch 1, Epoch 2, Epoch 3
Metric	Base	Epoch 1	Epoch 2	Epoch 3
Optimizer step	0	38	76	114
Mean verifier reward	0.0857	0.1601	0.1912	0.2838
Passes all implemented checks	5 / 105	10 / 105	9 / 105	16 / 105
Strict acceptance rate	4.8%	9.5%	8.6%	15.2%
Zero-reward outputs	83	78	74	61

The final reward is 3.31× the base reward, an absolute increase of 0.1981. These are results from one run, without repeated-seed uncertainty estimates or human listening scores. Positive verifier reward does not guarantee full instruction compliance.

The main remaining errors are excessive composition length, incorrect beats per bar, and inconsistent voices. The generation guard checks completed lines and can miss a runaway unfinished line. These limitations matter alongside the reward improvement.

Evidence: exact summary, 420 per-prompt validation records, and saved training settings and IDs.

Training configuration

Table with columns: Setting, Recorded configuration
Setting	Recorded configuration
Student	`Qwen/Qwen3.8-27B`
Student revision	`1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`
Teacher	`gpt-6.1-sol`, low reasoning effort
Training data	607 accepted teacher responses
Objective	Completion-only supervised cross-entropy
Quantization	4-bit NF4, double quantization; BF16 compute
LoRA	Rank 16, alpha 32, dropout 0.05
Epochs / optimizer updates	3 / 114
Microbatch / gradient accumulation	1 / 16; nominal effective batch 16
Learning rate / schedule	`1e-4` / cosine, one warmup step
Sequence length	11,008 selected for this run; configured ceiling 16,384
Checkpointing	Epoch boundaries; gradient checkpointing enabled

The longest training sequence contained 11,000 tokens. The trainer selected a padded length of 11,008 within the configured ceiling. See config.json and the saved run settings for their distinct roles.

GPUs, training time, and inference

The Modal architecture separates training and inference into two independently scheduled GPU roles, each requesting one NVIDIA H100. The inference device was recorded as NVIDIA H100 80GB HBM3. Teacher collection is coordinated on CPU and uses a hosted teacher service; its provider-side GPU hardware is not recorded here.

This is a single-GPU QLoRA training job plus a separate single-GPU inference job. It does not use DDP, FSDP, or multi-GPU tensor parallelism. The coordinator can overlap training and evaluation, but the saved worker timestamps for this run show sequential execution: inference began after the recorded training workers ended. Two GPU roles therefore does not mean two GPUs were simultaneously active in these measurements.

Table with columns: Measurement, Recorded value, Scope
Measurement	Recorded value	Scope
Final trainer runtime	40m 03s	`train_runtime` in the final saved trainer result
Training worker invocation 1	41m 34s	Full recorded invocation
Training worker invocation 2	41m 21s	Full recorded invocation
Total recorded training GPU time	1h 22m 55s	Sum of both training invocations
Inference worker time	4h 35m 33s	Full recorded inference invocation
Total recorded GPU time	5h 58m 28s	Training + inference; about 5.97 GPU-hours
vLLM startup	5m 54s	Included in inference worker time
Peak training memory allocated / reserved	44.77 / 48.30 GiB	PyTorch CUDA allocator metrics

Worker times include loading, scoring, and waiting inside the recorded invocation; they exclude container provisioning before function entry. The training total includes both invocations rather than presenting the final trainer metric as the entire compute budget. Runtime is not a provider billing total, and inference time is not a pure decoding throughput benchmark. Raw timestamps and measurements are in evaluation/runtime.json.

Inference configuration

Engine: vLLM 0.31.0, Torch 2.13.0+cu130, native FP8 per-tensor weights, LoRA enabled.
Queue capacity: up to 64 sequences on one H100, with prefix caching.
Context / batch token budget: 12,288 / 4,096 tokens.
Decoding: greedy generation, up to 8,192 new tokens, seed 42.
GPU memory utilization setting: 0.90.
Protocol: base model and epoch checkpoints; grouped queue with the structural guard.

The 64-sequence setting is serving concurrency, not a GPU count. The evaluated artifact namespace is recorded in evaluation/summary.json.

Intended use and limitations

Use this checkpoint for research into symbolic music generation, response distillation, and verifier-guided data selection. It outputs ABC text, not audio; render valid ABC with an appropriate synthesizer or notation tool.

Only 16 of 105 final validation outputs passed all implemented checks, and 61 scored zero. Check bar counts, beats, voices, syntax, and the requested structure before using an output. A positive reward does not establish musical quality, originality, or complete instruction compliance. No test-set score or human listening study is reported.

License and attribution

The adapter is released under Apache-2.0 for rights the publisher can grant. The base model's card declares Apache-2.0. Training prompts originate from XiaomiMiMo/MiMo-V2.6-RL-oss. See the dataset card and NOTICE for provenance and credits. The license does not override applicable third-party or provider terms; unrestricted reuse of teacher outputs has not been established.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

suunny

Model Tree

Base

Qwen/Qwen3.8-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality