GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Ankush0845
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Ankush0845
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Scored on the project's 45-question finance quiz, greedy decoding, base model scored in the same session for a like-for-like comparison:
| model | score |
|---|---|
| Qwen2.5-1.5B-Instruct (base) | 38/45 (84.44%) |
| base + this adapter | 40/45 (88.89%) |
| the project's own 101M from-scratch model, for contrast | 1-5/45 |
Training loss 1.905, validation loss 1.813.
Two points of honesty about that +4.45: it is two extra questions out of 45, so the adapter is a modest refinement rather than a transformation - most of the quality is Qwen's. And the scorer matches keywords and numbers, so it rewards phrasing as well as correctness.
| method | LoRA, r=16, alpha=32, dropout 0.05 |
| target modules | q, k, v, o, gate, up, down projections |
| trainable | 18.5M of 1.56B parameters (1.18%) |
| data | 11,990 finance instruction examples (of 38,477 built) |
| sources | finance-alpaca, Investopedia instruction set, Dolly-15k |
| epochs / LR | 1 epoch, 2e-4, cosine, fp16 |
| hardware | one Kaggle T4 |
Loss is taken on the answer tokens only; prompts are masked. Every one of the 45 evaluation questions was excluded from the training data by an automated leakage check (one match was found and removed).
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen2.5-1.5B-Instruct"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, dtype="float16", device_map="auto")model = PeftModel.from_pretrained(model, "Ankush0845/qwen2.5-1.5b-finance-lora") messages = [ {"role": "system", "content": "You are a financial analyst assistant. Answer concisely and correctly."}, {"role": "user", "content": "What is EBITDA?"},]text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=128)print(tok.decode(out[0], skip_special_tokens=True))Prompt it with the Qwen chat template, as above - that is the format it was tuned in.
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen2.5-1.5B-Instruct"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, dtype="float16", device_map="auto")model = PeftModel.from_pretrained(model, "Ankush0845/qwen2.5-1.5b-finance-lora") messages = [ {"role": "system", "content": "You are a financial analyst assistant. Answer concisely and correctly."}, {"role": "user", "content": "What is EBITDA?"},]text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=128)print(tok.decode(out[0], skip_special_tokens=True))