GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
kshitizjangra
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
kshitizjangra
Model Tree
Qwen/Qwen2-VL-2B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Single-shot OCR of a tightly cropped image containing one handwritten numeric/short value. Output is the value only, no prose.
Prompt (used at training and inference):
text
Read the handwritten value. Output only the value.| Base model | Qwen/Qwen2-VL-2B-Instruct |
| Method | PEFT LoRA |
Rank (r) | 16 |
lora_alpha | 32 |
lora_dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj |
| Task type | CAUSAL_LM |
| Data | OMR Part-C cell crops, JSONL splits (80/20 train/eval) |
| Starting adapter | Earlier Part-D LoRA (continued fine-tune) |
python
from peft import PeftModelfrom transformers import AutoProcessor, Qwen2VLForConditionalGenerationfrom PIL import Imageimport torch BASE = "Qwen/Qwen2-VL-2B-Instruct"ADAPTER = "kshitizjangra/qwen2vl-omr-lora-partc" processor = AutoProcessor.from_pretrained(BASE)model = Qwen2VLForConditionalGeneration.from_pretrained(BASE, torch_dtype=torch.float16, device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER)model.eval() image = Image.open("crop.jpg").convert("RGB")messages = [{ "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": "Read the handwritten value. Output only the value."}, ],}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=16, do_sample=False)print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0].strip())| File | Purpose |
|---|---|
adapter_model.safetensors | LoRA weights |
adapter_config.json | PEFT config |
tokenizer.json, tokenizer_config.json, chat_template.jinja | Tokenizer + chat template |
processor_config.json | Image/text processor |
marks_obtained cells. Other handwriting domains (full-page free-form, non-English script, very long sequences) are out of scope.Source code for cropping, dataset building, training, and inference lives at: https://github.com/kshitizjangra/omr_validator
from peft import PeftModelfrom transformers import AutoProcessor, Qwen2VLForConditionalGenerationfrom PIL import Imageimport torch BASE = "Qwen/Qwen2-VL-2B-Instruct"ADAPTER = "kshitizjangra/qwen2vl-omr-lora-partc" processor = AutoProcessor.from_pretrained(BASE)model = Qwen2VLForConditionalGeneration.from_pretrained(BASE, torch_dtype=torch.float16, device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER)model.eval() image = Image.open("crop.jpg").convert("RGB")messages = [{ "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": "Read the handwritten value. Output only the value."}, ],}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=16, do_sample=False)print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0].strip())