Contents
adapter_model.safetensors + adapter_config.json — PEFT adapter
adapter.gguf — same adapter converted for llama.cpp (convert_lora_to_gguf.py)
tokenizer.json, tokenizer_config.json, chat_template.jinja
LoRA: r=16, alpha=32, targets q/k/v/o/gate/up/down. Base: Qwen/Qwen3.5-0.8B-Base
(Apache-2.0). Trained on Kaggle (see kaggle/train.py).
Eval (200-row golden, student vs incumbent gpt-5-nano)
Table with columns: metric, student, incumbent| metric | student | incumbent |
|---|
| field-F1 n/a | acc 83.5% / macro-F1 0.802 | acc 73.5% / 0.686 |
| $/1k req | ~$0.014–0.024 | $0.11–0.28 |
Golden = official Banking77 TEST split (zero train overlap, enforced).
Serving
Per-request adapter select (llama.cpp): {"lora":[{"id":<sorted-adapter-idx>,"scale":1.0}],"chat_template_kwargs":{"enable_thinking":false}} — see modal/distill_serve.py.
Training data
Shaq2/distill-intent-routing — training-data license: cc-by-4.0
(provenance + per-source licenses in that dataset card and
data/SOURCES.md). Teacher labels generated by cerebras:gpt-oss-120b.