GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
trishpurkait
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
trishpurkait
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Evaluated on a 470-example held-out test set with an independent LLM judge (Qwen3-Next-80B), scoring 0–10 on four dimensions:
| Metric | Base | Fine-tuned |
|---|---|---|
| Factual correctness | 3.88 | 7.11 |
| Schema following | 3.34 | 8.66 |
| Field completeness | 3.71 | 6.91 |
| Hallucination control | 2.31 | 7.52 |
| Overall | 2.98 | 7.40 |
The base model frequently invents fields and values not present in the source OCR text (e.g. a fabricated email address, a fabricated timestamp); the fine-tuned model follows the target schema and defaults to null instead of guessing. See the repo's examples.md for annotated before/after cases.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModelBASE_MODEL = "Qwen/Qwen2.5-1.5B-Instruct"ADAPTER = "your-username/billstructai-qwen-lora"bnb_config = BitsAndBytesConfig(load_in_4bit=True,bnb_4bit_quant_type="nf4",bnb_4bit_compute_dtype=torch.float16,bnb_4bit_use_double_quant=True,)tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, quantization_config=bnb_config, device_map="auto")model = PeftModel.from_pretrained(base_model, ADAPTER)model.eval()SYSTEM_PROMPT = """You are a precise invoice information extraction assistant.Your task is to extract structured invoice data from noisy OCR text.Return only valid JSON.Do not explain.Do not add markdown.Use null for missing fields.Do not hallucinate values that are not present in the OCR text."""ocr_text = "PASTE OCR TEXT HERE"messages = [{"role": "system", "content": SYSTEM_PROMPT},{"role": "user", "content": f"Extract invoice information from the OCR text below and return only valid JSON.\n\nOCR_TEXT:\n{ocr_text}"},]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device)with torch.no_grad():output = model.generate(**inputs, max_new_tokens=768, do_sample=False, pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
json
{"vendor": {"name": null, "address": null, "tax_id": null, "phone": null, "email": null, "website": null},"customer": {"name": null, "address": null, "customer_id": null},"invoice_number": null,"invoice_date": null,"due_date": null,"purchase_order": null,"line_items": [{"description": null, "quantity": null, "unit": null, "unit_price": null, "amount": null, "tax_rate": null}],"subtotal": null,"tax_amount": null,"discount": null,"total_amount": null,"amount_paid": null,"balance_due": null,"currency": null,"payment_terms": null,"payment_method": null,"notes": null}
| LoRA rank | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Quantization | 4-bit NF4, double quant |
| Epochs | 2 |
| Effective batch size | 8 (1 × 8 grad. accumulation) |
| Learning rate | 2e-4, cosine schedule |
| Selected checkpoint | step 900 / 940 (lowest eval_loss) |