GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
bmarlin96
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
bmarlin96
Model Tree
TinyLlama/TinyLlama-1.1B-Chat-v1.0
Input Modalities
Output Modalities
Supported Functionality
The base model is TinyLlama/TinyLlama-1.1B-Chat-v1.0, loaded with a new six-way sequence-classification head. The base weights were frozen, and LoRA adapters were trained on the four attention projections in every decoder block (q_proj, k_proj, v_proj, o_proj) with rank r = 8 and lora_alpha = 16. Training ran for 3 epochs in fp32 on a single Colab T4 GPU and took a few minutes. The adapter files are a few megabytes, compared with about 4.4 GB for the full base model in fp32.
The model predicts one of six categories. Confirm that this order matches the id2label mapping used in training before relying on it.
| id | label | meaning |
|---|---|---|
| 0 | shortage | A supplier reports it cannot fill an order or a part is out of stock |
| 1 | delivery | Shipping, tracking, delay, or arrival notices |
| 2 | recall | Product recalls or safety notices on supplied parts |
| 3 | billing | Invoices, payment terms, and billing disputes |
| 4 | marketing | Promotions, newsletters, and sales outreach |
| 5 | confirmation | Order and receipt confirmations |
The adapter is meant for classroom demonstration of parameter-efficient fine-tuning. It shows how a small model with a few megabytes of trained weights can route messages that a zero-shot classifier handles poorly. It can also serve as the hosted model that an n8n workflow calls to triage incoming messages in a class exercise.
It is not intended for routing real business email. It has only seen 270 short synthetic messages written in a narrow set of phrasings, and it has no option to say that a message fits none of the six categories. Any real deployment would need training on real messages, a category for messages that need a person, and human review of low-confidence predictions.
The training data is meridian_inbox.csv, a generated set of 360 messages, 60 in each of the six categories. Each category was written in 12 phrasing templates. The last 3 templates in every category were held out as the test set, which gives 270 training messages and 90 test messages. Because the split is by phrasing rather than by random row, the test messages use wording the model never saw during training. The file is regenerated with a seeded script (make_meridian_inbox.py).
The notebook compares three approaches on the 90 held-out test messages.
| Approach | Test accuracy |
|---|---|
| Zero-shot baseline (facebook/bart-large-mnli with the six label strings) | [fill in] |
| TinyLlama with an untrained six-way head | [fill in] (about chance, 1 in 6) |
| TinyLlama with this LoRA adapter | [fill in] |
The notebook also runs the twelve inbox notices from Lesson 2 (M-01 to M-12) through the tuned model. M-12 is deliberately ambiguous and has no single correct label, so it is a useful check on how the model behaves when a message does not fit cleanly.
The data is synthetic and templated, so accuracy on the test set overstates how the model would do on real email, which is longer, messier, and often covers more than one topic. The model always picks one of the six labels, even for messages that belong to none of them. Ambiguous messages such as M-12 will still receive a confident-looking label. The base model is small and was trained mostly on English web text, so non-English messages are out of scope.
TinyLlama has no padding token, so the end-of-sequence token is used for padding. Replace your-username with the account that holds the adapter.
python
import torchfrom transformers import AutoTokenizer, AutoModelForSequenceClassificationfrom peft import PeftModel base_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"adapter_id = "your-username/meridian-router-lora"labels = ["shortage", "delivery", "recall", "billing", "marketing", "confirmation"] tokenizer = AutoTokenizer.from_pretrained(base_id)tokenizer.pad_token = tokenizer.eos_token base = AutoModelForSequenceClassification.from_pretrained(base_id, num_labels=6)base.config.pad_token_id = tokenizer.pad_token_id model = PeftModel.from_pretrained(base, adapter_id)model.eval() text = "Our supplier says the M8 hex bolts are backordered until next month."inputs = tokenizer(text, return_tensors="pt")with torch.no_grad(): probs = model(**inputs).logits.softmax(-1)[0] pred = probs.argmax().item()print(labels[pred], round(probs[pred].item(), 3))If you use this adapter in teaching material, please cite the base model:
bibtex
@misc{zhang2024tinyllama, title = {TinyLlama: An Open-Source Small Language Model}, author = {Peiyuan Zhang and Guangtao Zeng and Tianduo Wang and Wei Lu}, year = {2024}, eprint = {2401.02385}, archivePrefix = {arXiv}}Ben [last name], Information Systems, VCU School of Business.
import torchfrom transformers import AutoTokenizer, AutoModelForSequenceClassificationfrom peft import PeftModel base_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"adapter_id = "your-username/meridian-router-lora"labels = ["shortage", "delivery", "recall", "billing", "marketing", "confirmation"] tokenizer = AutoTokenizer.from_pretrained(base_id)tokenizer.pad_token = tokenizer.eos_token base = AutoModelForSequenceClassification.from_pretrained(base_id, num_labels=6)base.config.pad_token_id = tokenizer.pad_token_id model = PeftModel.from_pretrained(base, adapter_id)model.eval() text = "Our supplier says the M8 hex bolts are backordered until next month."inputs = tokenizer(text, return_tensors="pt")with torch.no_grad(): probs = model(**inputs).logits.softmax(-1)[0] pred = probs.argmax().item()print(labels[pred], round(probs[pred].item(), 3))@misc{zhang2024tinyllama, title = {TinyLlama: An Open-Source Small Language Model}, author = {Peiyuan Zhang and Guangtao Zeng and Tianduo Wang and Wei Lu}, year = {2024}, eprint = {2401.02385}, archivePrefix = {arXiv}}