GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
affanaamin
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
affanaamin
Model Tree
Qwen/Qwen2.5-VL-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
In Pakistan and across South Asia, trillions of rupees in municipal, provincial, and corporate disclosures are published exclusively as non-searchable, multi-generation scanned PDFs:
.sql and Schema.org JSON-LD without sending sensitive records to external cloud APIs.| Attribute | affanaamin/Dwanchi-VLM-7B-v0.1 (Canary Release) | affanaamin/Dwanchi-VLM-7B-v0.2 (Production Multi-Task) |
|---|---|---|
| Base Architecture | Qwen/Qwen2.5-VL-7B-Instruct (4-bit BNF4) | Qwen/Qwen2.5-VL-7B-Instruct (4-bit BNF4) |
| QLoRA Rank / Alpha | r = 16, lora_alpha = 16 | r = 32, lora_alpha = 32 |
| Trainable Parameters | 51,521,536 / 8.34B (0.62%) | 103,043,072 / 8.40B (1.23%) |
| Dataset Scale | 70 scanned municipal pages | 294 authentic pages / 417 multi-task samples |
| Trained Tasks | Structured Table Extraction | 1. Table Extraction (294) • 2. Fiscal Variance Q&A (74) • 3. 2D Visual Grounding (49) |
| Max Image Resolution | 1280 * 28 * 28 (1,003,520 px) | 1536 * 28 * 28 (1,204,224 px) |
| Optimization Steps | 60 steps (adamw_8bit, lr=2.0e-4) | 180 steps (adamw_8bit, cosine LR 2.0e-4, 1.72 epochs) |
| Final Training Loss | 0.7689 (437.7s on Kaggle Dual T4) | 0.2307 (2511.5s on Kaggle Dual T4 — 70.0% loss reduction) |
Dwanchi-VLM ships with Dwanchi-Bench (benchmark/hallucination_bench.py), an open red-teaming benchmark evaluating AI numerical hallucination on developing-world public budgets across 417 multi-task items:
۰-۹) and Eastern Arabic (٠-٩) numerals.Allocated - Actual == Variance), and 2D Table Bounding Box IoU ([ymin, xmin, ymax, xmax]).100.0% verified arithmetic reconciliation and 100.0% bilingual Urdu+English field completeness across all 417 benchmark records (python -m benchmark.hallucination_bench).python
import torchfrom unsloth import FastVisionModelfrom PIL import Image model, tokenizer = FastVisionModel.from_pretrained( "affanaamin/Dwanchi-VLM-7B-v0.2", load_in_4bit=True,)FastVisionModel.for_inference(model) image = Image.open("haripur_council_p000.png").convert("RGB")instruction = ( "Extract all fiscal line items from this Pakistani municipal budget table. " "Return valid JSON with keys: province, department, and table (scheme_code, " "scheme_name_english, scheme_name_urdu, budget_estimates, revised_estimates, actual_expenditures).") messages = [ { "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": instruction}, ], }] input_text = tokenizer.apply_chat_template(messages, add_generation_prompt=True)inputs = tokenizer(image, input_text, add_special_tokens=False, return_tensors="pt").to("cuda") with torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=1024, use_cache=True, temperature=0.1) print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))python
import torchfrom transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom peft import PeftModelfrom PIL import Image base_model_name = "Qwen/Qwen2.5-VL-7B-Instruct"processor = AutoProcessor.from_pretrained(base_model_name)base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained( base_model_name, torch_dtype=torch.bfloat16, device_map="auto",) model = PeftModel.from_pretrained(base_model, "affanaamin/Dwanchi-VLM-7B-v0.2")model.eval() image = Image.open("haripur_council_p000.png").convert("RGB")prompt = "<|user|><|image_1|>Extract the bilingual budget table into JSON format.<|end|><|assistant|>"inputs = processor(text=[prompt], images=[image], return_tensors="pt").to("cuda") with torch.no_grad(): generated_ids = model.generate(**inputs, max_new_tokens=1024) print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])bash
# Export validated SQLite .db, .sql dump, BiDi-isolated Markdown, accounting reconciliation & JSON-LDpython -m dwanchi_audit.cli --input data/processed/dataset.json --out-dir outputs/audit_bundle # Deterministically detect & crop header, table grid, official stamp/seal & summary footer from a scanpython -m dwanchi_audit.cli --crop-image demo/samples/haripur_council_p000.png --out-dir outputs/audit_bundle # Generate offline .gguf (Q4_K_M / Q8_0) build manifest & zero-temperature Ollama Modelfile (<8 GB VRAM)python scripts/export_gguf.py --out-dir outputs/gguf_bundle --quant q4_k_myaml
Training Recipe (v0.2 Production / v0.1 Canary): Base Model: "Qwen/Qwen2.5-VL-7B-Instruct" Quantization: "4-bit BNF4 (Unsloth)" PEFT LoRA Rank (r): 32 # (v0.1: 16) PEFT LoRA Alpha (alpha): 32 # (v0.1: 16) LoRA Dropout: 0.0 Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"] Vision & Language Layers Finetuned: true Optimizer: "adamw_8bit" Learning Rate: 2.0e-4 Learning Rate Scheduler: "cosine" Training Steps: 180 # (v0.1: 60) Batch Size per Device: 1 Gradient Accumulation Steps: 4 Effective Batch Size: 4 Image Token Constraints: Min Pixels: 200,704 (256 * 28 * 28) Max Pixels: 1,204,224 (1536 * 28 * 28) Final Training Loss: 0.2307Technical Verification Summary (NLnet Foundation, Trelis AI, Bagel Fund, BlueDot Impact, Manifund, Emergent Ventures, Z Fellows, Mozilla Foundation, Open Technology Fund):
CC-BY-4.0): Built from 294 real official Khyber Pakhtunkhwa (Revenue & Estate, Developmental & Current) and Tehsil Municipal Administration (TMA) Haripur budget gazette pages (affanaamin/dwanchi-vlm-dataset), with zero proprietary closed-API synthetic distillation (PROVENANCE.md).Apache-2.0):
dwanchi_audit/layout_parser.py: Horizontal ink-density projection and chromatic seal detection deterministically isolate bilingual_header, ledger_table_grid, official_stamp_or_seal, and summary_total_footer blocks, while tile_high_res_ledger() slices tall pages along whitespace valleys so <8 GB VRAM laptops never bisect Urdu Nastaliq descenders.dwanchi_audit/bidi_accounting.py: Wraps RTL Urdu Nastaliq cells in Unicode RLI..PDI (U+2067..U+2069) and LTR PKR figures in LRI..PDI (U+2066..U+2069) to prevent column-order inversion, and deterministically reconciles horizontal row variances and vertical column totals (flagging cell mismatches or 10x dropped/extra-zero smudge hallucinations)..safetensors + .gguf): Zero cloud API dependency at inference time. Exports directly to normalized SQLite .db, .sql transaction dumps, CSV, strict Schema.org JSON-LD (Dataset and Report), and offline Q4_K_M / Q8_0 .gguf Ollama bundles (scripts/export_gguf.py).Dwanchi-Bench) & Municipal Execution Tracking: Reproducible red-teaming harness (benchmark/hallucination_bench.py) measuring numerical hallucination, column bleed, RTL digit inversion, and variance reconciliation alongside municipal allocation tracking (dwanchi_audit/municipal_audit.py).bibtex
@misc{dwanchi_vlm_2026, author = {Aamin, Affan}, title = {Dwanchi-VLM-7B: Open Bilingual Vision-Language Model and Hallucination Benchmark for South Asian Public Finance Auditing}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/affanaamin/Dwanchi-VLM-7B-v0.2}},}Engineered by Affan Aamin (Dwanchi Studios, Peshawar, Pakistan) with Unsloth, Qwen2.5-VL, and PyMuPDF.
import torchfrom unsloth import FastVisionModelfrom PIL import Image model, tokenizer = FastVisionModel.from_pretrained( "affanaamin/Dwanchi-VLM-7B-v0.2", load_in_4bit=True,)FastVisionModel.for_inference(model) image = Image.open("haripur_council_p000.png").convert("RGB")instruction = ( "Extract all fiscal line items from this Pakistani municipal budget table. " "Return valid JSON with keys: province, department, and table (scheme_code, " "scheme_name_english, scheme_name_urdu, budget_estimates, revised_estimates, actual_expenditures).") messages = [ { "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": instruction}, ], }] input_text = tokenizer.apply_chat_template(messages, add_generation_prompt=True)inputs = tokenizer(image, input_text, add_special_tokens=False, return_tensors="pt").to("cuda") with torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=1024, use_cache=True, temperature=0.1) print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))import torchfrom transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom peft import PeftModelfrom PIL import Image base_model_name = "Qwen/Qwen2.5-VL-7B-Instruct"processor = AutoProcessor.from_pretrained(base_model_name)base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained( base_model_name, torch_dtype=torch.bfloat16, device_map="auto",) model = PeftModel.from_pretrained(base_model, "affanaamin/Dwanchi-VLM-7B-v0.2")model.eval() image = Image.open("haripur_council_p000.png").convert("RGB")prompt = "<|user|><|image_1|>Extract the bilingual budget table into JSON format.<|end|><|assistant|>"inputs = processor(text=[prompt], images=[image], return_tensors="pt").to("cuda") with torch.no_grad(): generated_ids = model.generate(**inputs, max_new_tokens=1024) print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])# Export validated SQLite .db, .sql dump, BiDi-isolated Markdown, accounting reconciliation & JSON-LDpython -m dwanchi_audit.cli --input data/processed/dataset.json --out-dir outputs/audit_bundle # Deterministically detect & crop header, table grid, official stamp/seal & summary footer from a scanpython -m dwanchi_audit.cli --crop-image demo/samples/haripur_council_p000.png --out-dir outputs/audit_bundle # Generate offline .gguf (Q4_K_M / Q8_0) build manifest & zero-temperature Ollama Modelfile (<8 GB VRAM)python scripts/export_gguf.py --out-dir outputs/gguf_bundle --quant q4_k_mTraining Recipe (v0.2 Production / v0.1 Canary): Base Model: "Qwen/Qwen2.5-VL-7B-Instruct" Quantization: "4-bit BNF4 (Unsloth)" PEFT LoRA Rank (r): 32 # (v0.1: 16) PEFT LoRA Alpha (alpha): 32 # (v0.1: 16) LoRA Dropout: 0.0 Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"] Vision & Language Layers Finetuned: true Optimizer: "adamw_8bit" Learning Rate: 2.0e-4 Learning Rate Scheduler: "cosine" Training Steps: 180 # (v0.1: 60) Batch Size per Device: 1 Gradient Accumulation Steps: 4 Effective Batch Size: 4 Image Token Constraints: Min Pixels: 200,704 (256 * 28 * 28) Max Pixels: 1,204,224 (1536 * 28 * 28) Final Training Loss: 0.2307@misc{dwanchi_vlm_2026, author = {Aamin, Affan}, title = {Dwanchi-VLM-7B: Open Bilingual Vision-Language Model and Hallucination Benchmark for South Asian Public Finance Auditing}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/affanaamin/Dwanchi-VLM-7B-v0.2}},}