GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
minemaster01
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
minemaster01
Model Tree
Qwen/Qwen2.5-VL-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
train_floorplan_vlm.py — MODEL_ID, HUB_MODEL_ID, OUTPUT_DIR, DATA_DIR, epochs, LR, LoRA layout, SYSTEM_PROMPT / USER_PROMPT (use these strings at inference).If you are working inside a clone of your training repo, the same script may exist locally as train_floorplan_vlm.py.
Inference / loading only:
bash
pip install torch torchvision transformers peft accelerate pillow
Full SFT training (matches script docstring; includes data prep deps):
bash
pip install torch torchvision transformers trl peft datasets accelerate shapely Pillow lxml numpy tqdm huggingface_hub# optional GPU attention: pip install flash-attn
Use Hub IDs (recommended):
python
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom peft import PeftModelBASE = "Qwen/Qwen2.5-VL-3B-Instruct"ADAPTER = "mudasir13cs/qwen25-vl-3b-floorplan-sft"processor = AutoProcessor.from_pretrained(BASE)model = Qwen2_5_VLForConditionalGeneration.from_pretrained(BASE, torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER)model.eval()
If you cloned weights into this folder locally: set ADAPTER = "./floorplan-vlm-sft" (or an absolute path) instead of the Hub repo id.
This stage defines the recommended prompts: the system message embeds the output JSON schema. Keep SYSTEM_PROMPT and USER_PROMPT aligned with train_floorplan_vlm.py constants.
User text: “Vectorize this floor plan into structured JSON with all walls, doors, windows, and rooms.”
End-to-end pattern (matches the inference test at the end of train_floorplan_vlm.py):
python
import json, re, torchfrom PIL import ImageSYSTEM_PROMPT = ("You are a floor plan vectorization expert. Extract wall, door, window geometry ""from floor plan images into structured JSON.\n\n""Output ONLY valid JSON with this schema:\n"'{"walls":[{"id":"wall_N","start":[x,y],"end":[x,y],"thickness":T,"curvature":0,''"openings":[{"type":"door"|"window","center":D,"width":W}]}],''"rooms":[{"label":"room_type","walls":["wall_N",...]}]}\n\n'"Coordinates normalized so longer image edge = 1024.")USER_PROMPT = "Vectorize this floor plan into structured JSON with all walls, doors, windows, and rooms."image = Image.open("plan.png").convert("RGB")messages = [{"role": "system", "content": [{"type": "text", "text": SYSTEM_PROMPT}]},{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": USER_PROMPT}]},]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[text], images=[image], return_tensors="pt", padding=True)inputs = {k: v.to(model.device) if hasattr(v, "to") else v for k, v in inputs.items()}with torch.no_grad():out = model.generate(**inputs, max_new_tokens=4096, do_sample=False)raw = processor.batch_decode(out[:, inputs.input_ids.shape[1] :], skip_special_tokens=True)[0]m = re.search(r"\{[\s\S]*\}", raw)plan = json.loads(m.group()) if m else None
Output shape: top-level walls (with optional openings) and rooms. Example JSON is documented under Output JSON Schema in the Manitocross training README.
huggingface-cli login if you push to Hub (PUSH_TO_HUB in script).train_floorplan_vlm.py: first run downloads CubiCasa5K from Zenodo into ./cubicasa_data (~5GB). Tune NUM_EPOCHS, MAX_SAMPLES, LEARNING_RATE, HUB_MODEL_ID, etc. in the configuration block at the top of that file.Qwen/Qwen2.5-VL-3B-InstructSFTTrainer / TRL; see script)walls, rooms, openings)Add your own run metadata: epochs, LR, seed, GPU, date, partial MAX_SAMPLES if applicable.
bibtex
@article{floorplanvlm2026,title={FloorplanVLM: A Vision-Language Model for Floorplan Vectorization},journal={arXiv preprint arXiv:2602.06507},year={2026}}
mudasir13cs/qwen25-vl-3b-floorplan-grpoMudasir — multimodal AI, VLM fine-tuning, retrieval/RAG research, and engineering; MS AI Convergence, 숭실대학교 — Soongsil University, Seoul. More credentials, publications, and projects: mudasir13cs.github.io