🤖 Model — Back2Struct-Image2SVG-7B
Table | |
|---|
| Base | Qwen/Qwen2.5-VL-7B-Instruct (7B) |
| Stage 1 — SFT | Supervised fine-tuning on StructHub image→SVG pairs |
| Stage 2 — GRPO | Group-Relative Policy Optimization (TRL) |
| Reward | Composite, render-gated: syntactic validity (must render) × perceptual fidelity (baseline-subtracted DINOv2 similarity between the rendered prediction and the target), encouraging compilability, length consistency, and structural/semantic faithfulness |
| Max output | 8,192 tokens · LoRA (r=64, α=128), merged into the released weights |
📚 Dataset — StructHub
Structured images paired with their editable SVG/XML source code.
Table with columns: Split, Description, # examples| Split | Description | # examples |
|---|
train | Full training set (all sources, all token lengths), benchmark held out | 86,102 |
benchmark | Held-out evaluation benchmark (starvector 569 + crawled 334 + nn_diagram 97) | 1,000 |
Fields: id (string), source (string), token_count (int32), image (embedded PNG),
svg (ground-truth SVG/XML). The benchmark is stratified by SVG length into easy
(≤2,048), medium (2,048–4,096), and hard (>4,096) tiers (333/333/334).
Results
StructHub benchmark, render-gated (a non-rendering prediction scores 0 on DINO/SSIM/GPT, 1 on LPIPS). SR = render success (%); GPT = 0–100 GPTScore.
Table with columns: Model, SR ↑, DINO ↑, LPIPS ↓, SSIM ↑, GPT ↑| Model | SR ↑ | DINO ↑ | LPIPS ↓ | SSIM ↑ | GPT ↑ |
|---|
| Overall | | | | | |
| Qwen3-VL-30B-A3B | 18.8 | 0.1566 | 0.9067 | 0.1201 | 13.29 |
| StarVector-8B | 31.4 | 0.2694 |
On successfully compiled outputs, the 7B model matches or beats frontier closed models:
Table with columns: Model, DINO ↑, LPIPS ↓, SSIM ↑, GPT ↑| Model | DINO ↑ | LPIPS ↓ | SSIM ↑ | GPT ↑ |
|---|
| Gemini-2.5-Pro | 0.8855 | 0.4648 | 0.6532 | 77.95 |
| GPT-5 | 0.8852 | 0.4350 | 0.6573 | 77.21 |
| Back2Struct (7B) | 0.8697 | 0.3896 | 0.6738 | |
Usage
Run the model
import torchfrom transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom qwen_vl_utils import process_vision_info model_id = "Pengyu965/Back2Struct-Image2SVG-7B"model = Qwen2_5_VLForConditionalGeneration.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(model_id) messages = [{"role": "user", "content": [ {"type": "image", "image": "diagram.png"}, {"type": "text", "text": "Convert this structured image into editable SVG/XML code."},]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)image_inputs, video_inputs = process_vision_info(messages)inputs = processor(text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt").to(model.device)generated = model.generate(**inputs, max_new_tokens=8192, do_sample=False)print(processor.batch_decode(generated[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])
Load the dataset
from datasets import load_datasetds = load_dataset("Pengyu965/StructHub")ex = ds["benchmark"][0]ex["image"].save("input.png"); print(ex["svg"][:500])
Intended use & limitations
- Intended use: recovering editable vector code from images of structured graphics.
- Not for: photographic / natural-image vectorization.
- Limitations: built on a general (non-code-specialized) VLM; outputs capped at 8,192
tokens; very long/dense diagrams (hard tier) remain the main failure mode (render reliability).
Licensing
Model weights are released under Apache-2.0 (inheriting the Qwen2.5-VL-7B-Instruct base).
The StructHub dataset aggregates multiple sources, including crawled open-domain graphics;
it is released for research use — review and respect the original content licenses.
(Confirm before wide distribution.)
Citation
@misc{yan2026back2structmakingstructuredimages, title={Back2Struct: Making Structured Images Editable Again}, author={Pengyu Yan and Yixin Wu and Yunjie Tian and David Doermann}, year={2026}, eprint={2609.37016}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2609.37016},}