xmanii

Mizan-v2-4B

README

License: apache-2.0

Images: what it can and can't do

Scored on about 1,600 images never used in training. "Balanced" weights every answer equally, so always saying "no" can't look good; the grey tick in the chart is that baseline.

Table with columns: Question, Balanced accuracy, Notes
Question	Balanced accuracy	Notes
Is there a person / vehicle / animal?	96–99%	camera-style frames, day and night
Vehicle type · animal type	93%	most prominent one, or "none"
Is a person / vehicle inside the red zone?	92% / 96%	draw a red box on the frame to mark the zone
How many people (0, 1, 2–3, 4–9, 10+)	86%
Bag · phone or laptop visible?	90% · 93%
Receipt total above X?	99%	real receipt photos (CORD)
Number of receipt line items	86%
Which product is on the line? (factory)	100%	VisA products
Is the product defective?	75%	useful for flagging, not for auto-rejecting; keep a human in the loop
Where is the defect?	not reliable	don't use it yet

Known issue: zone questions on night-mode frames. In about a quarter of the zone examples, a bug turned the red box grey (night mode was applied after the box was drawn). Zone questions are reliable on colour frames (correct on every colour test frame we checked) and weaker on grayscale ones. This is fixed in the data pipeline for the next version.

Limits. Training images were public photos (COCO, VisA, CORD) given a camera look: overlays, night mode, compression. They weren't real CCTV footage. Test it on a few hundred of your own frames before automating. Don't use it to identify people. It was trained only on presence and behaviour questions.

Text

Table with columns: Task (real Persian data), Mizan v2, Mizan 1.5, Mizan 1.0, Clef-Flash 9B, Jev (API)
Task (real Persian data)	Mizan v2	Mizan 1.5	Mizan 1.0	Clef-Flash 9B	Jev (API)
Food-review sentiment (SnappFood)	87.8	87.5	87.0	88.0	86.0
Product recommend (Digikala)	76.0	73.8	74.8	76.2	74.8
Aspect question in Persian (ParsiNLU)

The GGUF builds score 82.8% (Q8) and 82.7% (Q4) on the same benchmark.

leaderboard

Inference

llama.cpp (recommended, CPU or GPU)

Needs llama.cpp b11443 or newer.

bash

huggingface-cli download xmanii/Mizan-v2-4B-GGUF mizan-v2-4b-Q4_K_M.gguf mmproj-mizan-v2-4b-Q8_0.gguf --local-dir .# text + images; drop --mmproj for text onlyllama-server -m mizan-v2-4b-Q4_K_M.gguf --mmproj mmproj-mizan-v2-4b-Q8_0.gguf -c 8192 -b 8192 -ub 8192 -ngl 99 --port 8080

python

import base64, httpx frame = "data:image/jpeg;base64," + base64.b64encode(open("cam03.jpg", "rb").read()).decode()body = {    "state": {"منبع": "دوربین پارکینگ"},    "images": [frame],    "questions": {        "person_in_zone": {"type": "noul", "instructions": "آیا کسی داخل کادر قرمز هست؟"},        "vehicle": {"type": "choice", "instructions": "وسیله نقلیه غالب کدام است؟",                    "criteria": {"car": "سواری", "truck": "کامیون", "motorcycle": "موتور", "none": "هیچ‌کدام"}},    },}print(httpx.post("http://localhost:8080/v1/systemone", json=body, timeout=60).json()["answers"])

Text requests are the same without images.

Speed:

On an NVIDIA L4 (Q4): about 80 ms per text request and about 470 ms with an image, median.
Images through llama.cpp score about the same as the full model; see the GGUF card.
Measured on CPU only (Q4, Mizan 1.5, same architecture): about 3 s median per text request on 16 cores, and about 18 s on 2 cores. For real-time use, run it on a GPU; an entry-level 8 GB card is enough.

Python (transformers, text + images)

python

import json, sys, torchfrom huggingface_hub import snapshot_downloadfrom PIL import Image path = snapshot_download("xmanii/Mizan-v2-4B")sys.path.insert(0, path)from joint_schema_model import collate_records, encode_record, load_release_model model, processor = load_release_model(path, device="cuda")temps = json.load(open(f"{path}/unsloth_decision_config.json"))["temperature"]  # choice, score, noulrecord = {"state": "تصویر پیوست", "images": [Image.open("cam03.jpg")],          "questions": {"person": {"type": "noul", "instructions": "آیا کسی در تصویر هست؟"}}}encoded = encode_record(processor.tokenizer, record, processor=processor)logits = model(collate_records([encoded], processor.tokenizer.pad_token_id, torch.device("cuda")))[0]for q, z in zip(encoded.questions, logits):    t = {0: temps[2], 1: temps[0], 2: temps[1]}[q.question_type]  # calibrated temperature for this type    print(q.question_id, dict(zip(q.option_ids, (z.float() / t).softmax(-1).tolist())))

Unsloth's FastDecisionModel also loads this repo, for text only.

Training

Start: Mizan 1.5. Its text-decision skills come from Unsloth training: Persian business workflows, mined real mistakes and an English decision mixture.
v2 stage: a custom loop around the Clef reference code (joint_schema_model.py), because Unsloth's decision trainer doesn't pass images.
- LoRA r=32 on the language model, a 10× lower-rate LoRA on the vision merger and last 4 vision blocks, and the decision head.
- 500 steps on 1× H100 (59 min), with soft-label cross-entropy per question.
Data:
- Images (~3,000 rows):
  - COCO frames given a security-camera look (camera name and Persian timestamp, night mode, compression, red zone boxes), with Persian and English questions. Answers come from COCO's human annotations, and questions are skipped when tiny, cut-off or crowded objects make the answer uncertain.
  - VisA factory photos, normal and defective, with product names.
  - CORD real receipts, with totals and item counts from their parsed labels.

License

Apache-2.0. Image training data: COCO (commercial-use licenses only), VisA (CC BY 4.0), CORD (CC BY 4.0).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

xmanii

Model Tree

Base

xmanii/Mizan-1.5-4B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Images: what it can and can't do

Scored on about 1,600 images never used in training. "Balanced" weights every answer equally, so always saying "no" can't look good; the grey tick in the chart is that baseline.

Table with columns: Question, Balanced accuracy, Notes
Question	Balanced accuracy	Notes
Is there a person / vehicle / animal?	96–99%	camera-style frames, day and night
Vehicle type · animal type	93%	most prominent one, or "none"
Is a person / vehicle inside the red zone?	92% / 96%	draw a red box on the frame to mark the zone
How many people (0, 1, 2–3, 4–9, 10+)	86%
Bag · phone or laptop visible?	90% · 93%
Receipt total above X?	99%	real receipt photos (CORD)
Number of receipt line items	86%
Which product is on the line? (factory)	100%	VisA products
Is the product defective?	75%	useful for flagging, not for auto-rejecting; keep a human in the loop
Where is the defect?	not reliable	don't use it yet

Text

Table with columns: Task (real Persian data), Mizan v2, Mizan 1.5, Mizan 1.0, Clef-Flash 9B, Jev (API)
Task (real Persian data)	Mizan v2	Mizan 1.5	Mizan 1.0	Clef-Flash 9B	Jev (API)
Food-review sentiment (SnappFood)	87.8	87.5	87.0	88.0	86.0
Product recommend (Digikala)	76.0	73.8	74.8	76.2	74.8
Aspect question in Persian (ParsiNLU)

The GGUF builds score 82.8% (Q8) and 82.7% (Q4) on the same benchmark.

leaderboard

Inference

llama.cpp (recommended, CPU or GPU)

Needs llama.cpp b11443 or newer.

bash

huggingface-cli download xmanii/Mizan-v2-4B-GGUF mizan-v2-4b-Q4_K_M.gguf mmproj-mizan-v2-4b-Q8_0.gguf --local-dir .# text + images; drop --mmproj for text onlyllama-server -m mizan-v2-4b-Q4_K_M.gguf --mmproj mmproj-mizan-v2-4b-Q8_0.gguf -c 8192 -b 8192 -ub 8192 -ngl 99 --port 8080

python

import base64, httpx frame = "data:image/jpeg;base64," + base64.b64encode(open("cam03.jpg", "rb").read()).decode()body = {    "state": {"منبع": "دوربین پارکینگ"},    "images": [frame],    "questions": {        "person_in_zone": {"type": "noul", "instructions": "آیا کسی داخل کادر قرمز هست؟"},        "vehicle": {"type": "choice", "instructions": "وسیله نقلیه غالب کدام است؟",                    "criteria": {"car": "سواری", "truck": "کامیون", "motorcycle": "موتور", "none": "هیچ‌کدام"}},    },}print(httpx.post("http://localhost:8080/v1/systemone", json=body, timeout=60).json()["answers"])

Text requests are the same without images.

Speed:

On an NVIDIA L4 (Q4): about 80 ms per text request and about 470 ms with an image, median.
Images through llama.cpp score about the same as the full model; see the GGUF card.
Measured on CPU only (Q4, Mizan 1.5, same architecture): about 3 s median per text request on 16 cores, and about 18 s on 2 cores. For real-time use, run it on a GPU; an entry-level 8 GB card is enough.

Python (transformers, text + images)

python

import json, sys, torchfrom huggingface_hub import snapshot_downloadfrom PIL import Image path = snapshot_download("xmanii/Mizan-v2-4B")sys.path.insert(0, path)from joint_schema_model import collate_records, encode_record, load_release_model model, processor = load_release_model(path, device="cuda")temps = json.load(open(f"{path}/unsloth_decision_config.json"))["temperature"]  # choice, score, noulrecord = {"state": "تصویر پیوست", "images": [Image.open("cam03.jpg")],          "questions": {"person": {"type": "noul", "instructions": "آیا کسی در تصویر هست؟"}}}encoded = encode_record(processor.tokenizer, record, processor=processor)logits = model(collate_records([encoded], processor.tokenizer.pad_token_id, torch.device("cuda")))[0]for q, z in zip(encoded.questions, logits):    t = {0: temps[2], 1: temps[0], 2: temps[1]}[q.question_type]  # calibrated temperature for this type    print(q.question_id, dict(zip(q.option_ids, (z.float() / t).softmax(-1).tolist())))

Unsloth's FastDecisionModel also loads this repo, for text only.

Training

Start: Mizan 1.5. Its text-decision skills come from Unsloth training: Persian business workflows, mined real mistakes and an English decision mixture.
v2 stage: a custom loop around the Clef reference code (joint_schema_model.py), because Unsloth's decision trainer doesn't pass images.
- LoRA r=32 on the language model, a 10× lower-rate LoRA on the vision merger and last 4 vision blocks, and the decision head.
- 500 steps on 1× H100 (59 min), with soft-label cross-entropy per question.
Data:
- Images (~3,000 rows):
  - COCO frames given a security-camera look (camera name and Persian timestamp, night mode, compression, red zone boxes), with Persian and English questions. Answers come from COCO's human annotations, and questions are skipped when tiny, cut-off or crowded objects make the answer uncertain.
  - VisA factory photos, normal and defective, with product names.
  - CORD real receipts, with totals and item counts from their parsed labels.

License

Apache-2.0. Image training data: COCO (commercial-use licenses only), VisA (CC BY 4.0), CORD (CC BY 4.0).