anybody12345678

lab21-qwen35-triage-vi

README

License: apache-2.0

Verdict: FAILED

The gate is not "did the loss go down". It is: does the fine-tune beat the well-prompted base model on the task, without losing general capability?

target Δ +0.115 · regression Δ -0.622 (tolerance 0.020)

general capability regressed by 0.622 (tolerance 0.020). See deck §14.3 — add 1-5% replay data.

Four-group evaluation

Scored on 50 held-out tickets + 15 general-knowledge probes, greedy decode.

Table with columns: run, target, regression, format, latency (ms)
run	target	regression	format	latency (ms)
(a) base + naive prompt	0.000	0.622	0.000	1509
(b) base + optimized prompt	0.450	0.622	1.000	535
(c) LoRA fine-tune	0.565	0.000	1.000	720

(a) base + a naive prompt · (b) base + a carefully engineered prompt · (c) this adapter
target = per-field accuracy over the 4 triage fields · regression = keyword recall on general-knowledge questions the fine-tune must not break · format = required keys present

Misconfiguration autopsy — scored on the task, not on training loss

Table with columns: run, target, format, latency (ms)
run	target	format	latency (ms)
`correct`	0.565	1.000	720
`attn_only`	0.545	1.000	452
`wrong_lr`	0.185	0.800	1278

attn_only carries the same trainable-parameter budget as correct (rank raised to match), so it isolates placement from rank. wrong_lr changes exactly one number: the learning rate, from the LoRA scale back to the full-fine-tune scale.

Training

Table

Base	`Qwen/Qwen3.5-0.8B`
Placement	`text-linear` — every text-decoder linear layer, vision tower excluded
r / alpha	16 / 32
Trainable params	10,822,656
Learning rate	0.0001
Optimizer steps	58
Precision	bf16
Peak VRAM	3.15 GB

The loss mask was verified by decoding it back, not assumed: the answer is inside the supervised span (True) and the question is not (True).

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3.5-0.8B"
tok = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base_id, trust_remote_code=True,
                                             dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "anybody12345678/lab21-qwen35-triage-vi")

msgs = [
    {"role": "system", "content": "Phân loại ticket sau."},
    {"role": "user", "content": "Shop ơi, mình đặt bàn phím cơ mã đơn DH123456. "
                                 "Giao hàng chậm. Đã 3 ngày rồi. Nhờ shop kiểm tra."},
]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                               enable_thinking=False)
out = model.generate(**tok(text, return_tensors="pt").to(model.device),
                     max_new_tokens=160, do_sample=False)
print(tok.decode(out[0][len(tok(text)["input_ids"]):], skip_special_tokens=True))

The system prompt matters. This adapter was trained with "Phân loại ticket sau." as the system message and the bare ticket as the user turn. Send it a different scaffold and it is being asked to continue a prompt it never saw.

Limitations — read these before using it

General capability is not preserved. See the regression column. Trained on bare ticket → JSON with no instruction, the model learns that any input means "emit triage JSON", and it answers general-knowledge questions in JSON too. The remedy is 1–5% replay data; this adapter does not have it.
Trained on a synthetic, template-generated Vietnamese corpus, not production tickets. The label space is a local convention ("hoàn lại" is filed under doi_tra, not hoan_tien) and will not match yours.
Small base model, short training budget. This is a teaching artifact.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

anybody12345678

Model Tree

Base

Qwen/Qwen3.5-0.8B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality