GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
bibbbu
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
bibbbu
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
The base 1.5B model could not reliably emit valid structured output for this task (0% schema validity on the held-out evaluation set). Rather than scaling up the model or adding heavy post-processing, this adapter was trained via teacher–student distillation to fix the failure mode directly at the model level.
Results on a held-out evaluation set, base model vs. base + this adapter:
| Metric | Base Qwen2.5-1.5B-Instruct | + FinBot LoRA v2 |
|---|---|---|
| JSON schema validity | 0% | 100% |
| Safety compliance | 100% | 100% (maintained) |
| Mean output tokens | 638 | 150 (−77%) |
| Mean latency | 5,134 ms | 2,902 ms (−43%) |
Evaluated on 62 held-out examples balanced across English (21), Vietnamese (21), and Chinese (20), spanning planning, investment, and trading tasks. Latency measured per-example on an NVIDIA A100-80GB. Schema validity reached 100% in every language individually, not just in aggregate. Evaluation covers cascading JSON schema validation, safety/refusal behaviour (unsafe-claim screening, e.g. "guaranteed profit"), structured internal-analysis presence, and output efficiency.
SFTTrainer) on teacher–student distilled examples, filtered and validated against the FinBot output schema before training.16, alpha: 32, dropout: 0.05q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (attention + MLP)python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_id = "Qwen/Qwen2.5-1.5B-Instruct"adapter_id = "bibbbu/lora-qwen25-1p5b-finbot-v2" tokenizer = AutoTokenizer.from_pretrained(base_id)model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")model = PeftModel.from_pretrained(model, adapter_id) messages = [ {"role": "system", "content": "You are FinBot, a financial planning assistant. Respond with a single JSON object matching the FinBot recommendation schema."}, {"role": "user", "content": "Profile: age 30, income bucket B, risk tolerance: moderate, goal: retirement savings, horizon: 25 years, language: en. Generate a recommendation."},]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)output = model.generate(inputs, max_new_tokens=512)print(tokenizer.decode(output[0][inputs.shape[-1]:], skip_special_tokens=True))The adapter runs on CUDA and Apple Silicon (MPS). For the full system — deterministic dialogue state machine, slot parsing, multilingual i18n, PII bucketing, prompt-injection defence, and JSON validation/repair pipeline — see the FinBot repository.
Intended use: research and demonstration of structured-output fine-tuning for small, locally deployed LLMs; portfolio/educational use; as the recommendation component inside the FinBot system, where its output is schema-validated before display.
Limitations:
bibtex
@misc{vu2026finbotlora, title = {FinBot LoRA: Structured-Output Fine-Tuning of Qwen2.5-1.5B for a Multilingual Local Financial Assistant}, author = {Vu, Tuong Vy}, year = {2026}, url = {https://huggingface.co/bibbbu/lora-qwen25-1p5b-finbot-v2}}2e-4manifest.json.