mjusdda

lab21-2A202601343-qwen35-triage-vi

README

License: apache-2.0

Evaluation

Full evaluation used 50 target samples and 15 general-regression samples. Generation was greedy on a Colab NVIDIA T4.

Table with columns: Run, Target, Regression, JSON format, Latency (ms/sample)
Run	Target	Regression	JSON format	Latency (ms/sample)
Base + naive prompt	0.000	0.7578	0.000	3370.5
Base + optimized prompt	0.765	0.7578	1.000	1066.3
This LoRA adapter	0.970	0.6778	1.000	1511.4

The adapter improves target score by +0.205 over the optimized-prompt baseline, but regression drops by -0.080, beyond the allowed tolerance of -0.020. The formal verdict is therefore FAILED. This is an honest experimental result, not a claim that the adapter is production-ready. See submission/REPORT.md and the machine-readable files under results/.

Training configuration

Base model: unsloth/Qwen3.5-4B
Method: PEFT LoRA, text-decoder linear layers
Rank / alpha: r=16, alpha=32
Learning rate: 1e-4
Precision: fp16
Epochs / optimizer steps: 2 / 30
Trainable parameters: 32,464,896
Peak allocated VRAM: 12.01 GB
Training time: 987.6 seconds
Loss mask: assistant-only; prompt tokens are excluded

Usage

python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "unsloth/Qwen3.5-4B"
adapter_id = "mjusdda/lab21-2A202601343-qwen35-triage-vi"

tokenizer = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    trust_remote_code=True,
    dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)

ticket = "Shop ơi, đơn bàn phím cơ của tôi giao chậm quá. Nhờ kiểm tra gấp."
messages = [
    {"role": "system", "content": "Phân loại ticket sau."},
    {"role": "user", "content": ticket},
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=160, do_sample=False)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

Intended use

This adapter is intended for education, reproducibility, and experiments with Vietnamese ticket triage. Expected closed-vocabulary values are:

intent: doi_tra, van_chuyen, hoan_tien, san_pham_loi, hoi_thong_tin
urgency: cao, trung_binh, thap
sentiment: tieu_cuc, trung_tinh, tich_cuc
product: product name copied from the ticket

Limitations

The training corpus contains only 225 training tickets and is synthetic.
General regression fell from 0.7578 to 0.6778, indicating measurable specialization or forgetting.
Six of 50 target examples scored 0.75; the recurring error was predicting urgency=trung_binh when the correct value was thap.
Do not deploy as a general assistant or use for consequential customer decisions without broader data, replay training, and independent evaluation.
No inference provider is required for this submission; the adapter must be loaded with its base model.

Reproducibility and files

Code and report: GitHub repository
Full report: submission/REPORT.md
Metrics: results/verdict.json, results/runs.csv, and results/autopsy.json

License

The adapter follows the Apache-2.0 license declared by the base model. The accompanying lab code repository is separately licensed under MIT.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

mjusdda

Model Tree

Base

unsloth/Qwen3.5-4B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality