utsubo12

qwen3-vl-8b-netdsl-l2

Model description

The model translates a 256×256 semiconductor SEM image into a NetDSL-L2 program — a Domain-Specific Language describing Manhattan-routed circuit layouts as a sequence of CANVAS, WIRE2 and VIA commands. Rendering the generated DSL reproduces the input geometry as a binary mask, enabling controlled augmentation, parameter editing and downstream metrology.

Base model: Qwen/Qwen3-VL-8B-Instruct
Fine-tuning: full SFT — vision encoder, multimodal projector and language model all trainable
Training data: 18,900 synthetic (image, DSL) pairs, entirely procedurally generated; no real SEM image is used in training
Optimization: 3 epochs, batch 8 × grad-accum 12 (effective 96), LR 2.0e-5 cosine + 10 % warmup, weight decay 0.01, max grad norm 1.0, pure BF16 + gradient checkpointing
Hardware: single NVIDIA H200 (141 GB)
Final training loss: 0.2832
Chat template: qwen3_vl_nothink (no reasoning tokens emitted)

Intended use

Feed a binarized SEM image; the model emits NetDSL-L2 code. The binarization is what closes the sim-to-real gap — the model only ever saw binary images during training.

python
from PIL import Image
import numpy as np

# 1. resize to the model's canvas, THEN threshold. Both steps matter.
img = Image.open("test_normal_00169.jpg").convert("L").resize((256, 256), Image.LANCZOS)
binary = Image.fromarray(np.where(np.array(img) > 96, 255, 0).astype(np.uint8), "L")

The threshold is 96, applied to the resampled pixels. (§III-D of the paper prints "100" and omits the resize; the released data and code use 96 after a LANCZOS resize — that recipe reproduces the published ground-truth images pixel-for-pixel.)

Evaluation (MIIC, 1034 real SEM test images)

Mean ± std over executable outputs. Binary input (proposed) vs raw input (baseline):

Table with columns: Metric, Raw, Binary (ours)
Metric	Raw	Binary (ours)
IoU	0.2865 ± 0.0802	0.3619 ± 0.0882
Dice coefficient	0.4393 ± 0.0980	0.5256 ± 0.0912
BF1 @ 2 px	0.4054 ± 0.1106	0.4412 ± 0.1098
SkF1 @ 1 px	0.1276 ± 0.0960	0.1746 ± 0.1145
ASSD	4.7768 ± 3.4596	4.1327 ± 1.2757

Executable outputs: 1019 of 1034 for raw input (98.5 %). For binary input the evaluation run was interrupted and resumed, and its aggregate covers 986 attempted images, of which 957 were executable (97.1 % = 957/986). Recomputing over all 1005 surviving binary renders gives IoU 0.3628 / Dice 0.5265 and executability 1005/1034 = 97.2 % — the same conclusion. Details and per-sample data: utsubo12/miic-netdsl-eval.

Usage

The prompt matters. The model was fine-tuned on one fixed instruction; a paraphrase will degrade output badly. Use it verbatim:

python
import numpy as np, torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText

REPO = "utsubo12/qwen3-vl-8b-netdsl-l2"

INSTRUCTION = """You are given a binary SEM-like image. Generate the NetDSL-L2 program that reproduces this image.

NetDSL-L2 Syntax:
- CANVAS W H: Set canvas size (256x256)
- WIRE2 x0 y0 base_w cap_s_len cap_s_w cap_e_len cap_e_w segments: Manhattan wiring with optional dogbone caps
  - x0, y0: start position
  - base_w: line width
  - cap_s_len, cap_s_w: start cap length and width (0 for none)
  - cap_e_len, cap_e_w: end cap length and width (0 for none)
  - segments: H length or V length sequence
- VIA x y r: Circle at (x,y) with radius r"""

model = AutoModelForImageTextToText.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, device_map="cuda")
processor = AutoProcessor.from_pretrained(REPO)

img = Image.open("test_normal_00169.jpg").convert("L").resize((256, 256), Image.LANCZOS)
binary = Image.fromarray(np.where(np.array(img) > 96, 255, 0).astype(np.uint8), "L")

messages = [{"role": "user", "content": [
    {"type": "image", "image": binary.convert("RGB")},
    {"type": "text", "text": INSTRUCTION},
]}]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt",
    tokenize=True, return_dict=True,
).to(model.device)

out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
dsl_code = processor.batch_decode(
    out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]
print(dsl_code)

Render the predicted DSL with the companion repository:

python
from src.dsl2_runtime import render_program_l2
mask = render_program_l2(dsl_code)     # -> PIL.Image, 256x256, mode "L"
mask.save("reconstruction.png")

The numbers in the paper were produced through LLaMA-Factory's ChatModel with template="qwen3_vl_nothink" (see src/eval_dsl2_v3_miic_fulldata.py), not through the transformers path above. Use the repository scripts for exact reproduction.

Reproduce the paper's figures

bash
git clone https://github.com/YusukeO/eusipco2026-sem-dsl && cd eusipco2026-sem-dsl
pip install -r requirements.txt

python -m src.reproduce_paper_figures --mode render-only   # verify renderer, no GPU
python -m src.reproduce_paper_figures --mode full --model-dir utsubo12/qwen3-vl-8b-netdsl-l2

NetDSL-L2

markdown
CANVAS 256 256
WIRE2 0 12 8   0 0   0 0   H 256
WIRE2 12 0 10  0 0  30 14  V 60
VIA   35 40 6

WIRE2 x0 y0 base_w cap_s_len cap_s_w cap_e_len cap_e_w <segments> describes a wire with optional dogbone end-caps; <segments> is a chain of H <length> / V <length> relative moves. See §III-A of the paper — which writes the primitive as WIRE, while the concrete L2 token is WIRE2 (WIRE is the NetDSL-L1 keyword).

Limitations

Trained only on synthetic Manhattan-style layouts; non-Manhattan or analog layouts are out of distribution.
Real SEM images must be binarized (resize to 256×256, then global threshold 96) to obtain the reported numbers; raw grayscale input degrades quality substantially — that is the paper's point.
Output is capped at ~2048 tokens; very dense layouts may be truncated.
Reconstruction quality falls as pattern complexity (compressed DSL length) grows; see Fig. 5 of the paper. The correlation between compressed DSL size and residual error is r = 0.497.
Roughly 3 % of real images yield non-executable DSL and are excluded from the metrics.

Citation

bibtex
@inproceedings{ohtsubo2026bridging,
  author    = {Ohtsubo, Yusuke and Dohi, Kota and Yawata, Koichiro and
               Takeshita, Koki and Sasaki, Tatsuya},
  title     = {Bridging the Sim-to-Real Gap in Semiconductor Visual Program
               Synthesis via Input Binarization},
  booktitle = {Proceedings of the 34th European Signal Processing Conference (EUSIPCO)},
  year      = {2026},
  publisher = {EURASIP},
  note      = {Accepted; final citation/DOI to be updated upon publication}
}

License

MIT for both the code and these weights. The base model Qwen3-VL-8B-Instruct is subject to its own license; please review it.

Contact

Yusuke Ohtsubo — yusuke.ohtsubo.nb@hitachi.com

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

utsubo12

Model Tree

Base

Qwen/Qwen3-VL-8B-Instruct

Fine-tuned

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Model description

Base model: Qwen/Qwen3-VL-8B-Instruct
Fine-tuning: full SFT — vision encoder, multimodal projector and language model all trainable
Training data: 18,900 synthetic (image, DSL) pairs, entirely procedurally generated; no real SEM image is used in training
Optimization: 3 epochs, batch 8 × grad-accum 12 (effective 96), LR 2.0e-5 cosine + 10 % warmup, weight decay 0.01, max grad norm 1.0, pure BF16 + gradient checkpointing
Hardware: single NVIDIA H200 (141 GB)
Final training loss: 0.2832
Chat template: qwen3_vl_nothink (no reasoning tokens emitted)

Intended use

Feed a binarized SEM image; the model emits NetDSL-L2 code. The binarization is what closes the sim-to-real gap — the model only ever saw binary images during training.

python
from PIL import Image
import numpy as np

# 1. resize to the model's canvas, THEN threshold. Both steps matter.
img = Image.open("test_normal_00169.jpg").convert("L").resize((256, 256), Image.LANCZOS)
binary = Image.fromarray(np.where(np.array(img) > 96, 255, 0).astype(np.uint8), "L")

Evaluation (MIIC, 1034 real SEM test images)

Mean ± std over executable outputs. Binary input (proposed) vs raw input (baseline):

Table with columns: Metric, Raw, Binary (ours)
Metric	Raw	Binary (ours)
IoU	0.2865 ± 0.0802	0.3619 ± 0.0882
Dice coefficient	0.4393 ± 0.0980	0.5256 ± 0.0912
BF1 @ 2 px	0.4054 ± 0.1106	0.4412 ± 0.1098
SkF1 @ 1 px	0.1276 ± 0.0960	0.1746 ± 0.1145
ASSD	4.7768 ± 3.4596	4.1327 ± 1.2757

Usage

The prompt matters. The model was fine-tuned on one fixed instruction; a paraphrase will degrade output badly. Use it verbatim:

python
import numpy as np, torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText

REPO = "utsubo12/qwen3-vl-8b-netdsl-l2"

INSTRUCTION = """You are given a binary SEM-like image. Generate the NetDSL-L2 program that reproduces this image.

NetDSL-L2 Syntax:
- CANVAS W H: Set canvas size (256x256)
- WIRE2 x0 y0 base_w cap_s_len cap_s_w cap_e_len cap_e_w segments: Manhattan wiring with optional dogbone caps
  - x0, y0: start position
  - base_w: line width
  - cap_s_len, cap_s_w: start cap length and width (0 for none)
  - cap_e_len, cap_e_w: end cap length and width (0 for none)
  - segments: H length or V length sequence
- VIA x y r: Circle at (x,y) with radius r"""

model = AutoModelForImageTextToText.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, device_map="cuda")
processor = AutoProcessor.from_pretrained(REPO)

img = Image.open("test_normal_00169.jpg").convert("L").resize((256, 256), Image.LANCZOS)
binary = Image.fromarray(np.where(np.array(img) > 96, 255, 0).astype(np.uint8), "L")

messages = [{"role": "user", "content": [
    {"type": "image", "image": binary.convert("RGB")},
    {"type": "text", "text": INSTRUCTION},
]}]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt",
    tokenize=True, return_dict=True,
).to(model.device)

out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
dsl_code = processor.batch_decode(
    out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]
print(dsl_code)

Render the predicted DSL with the companion repository:

python
from src.dsl2_runtime import render_program_l2
mask = render_program_l2(dsl_code)     # -> PIL.Image, 256x256, mode "L"
mask.save("reconstruction.png")

The numbers in the paper were produced through LLaMA-Factory's ChatModel with template="qwen3_vl_nothink" (see src/eval_dsl2_v3_miic_fulldata.py), not through the transformers path above. Use the repository scripts for exact reproduction.

Reproduce the paper's figures

bash
git clone https://github.com/YusukeO/eusipco2026-sem-dsl && cd eusipco2026-sem-dsl
pip install -r requirements.txt

python -m src.reproduce_paper_figures --mode render-only   # verify renderer, no GPU
python -m src.reproduce_paper_figures --mode full --model-dir utsubo12/qwen3-vl-8b-netdsl-l2

NetDSL-L2

markdown
CANVAS 256 256
WIRE2 0 12 8   0 0   0 0   H 256
WIRE2 12 0 10  0 0  30 14  V 60
VIA   35 40 6

Limitations

Trained only on synthetic Manhattan-style layouts; non-Manhattan or analog layouts are out of distribution.
Real SEM images must be binarized (resize to 256×256, then global threshold 96) to obtain the reported numbers; raw grayscale input degrades quality substantially — that is the paper's point.
Output is capped at ~2048 tokens; very dense layouts may be truncated.
Reconstruction quality falls as pattern complexity (compressed DSL length) grows; see Fig. 5 of the paper. The correlation between compressed DSL size and residual error is r = 0.497.
Roughly 3 % of real images yield non-executable DSL and are excluded from the metrics.

Citation

bibtex
@inproceedings{ohtsubo2026bridging,
  author    = {Ohtsubo, Yusuke and Dohi, Kota and Yawata, Koichiro and
               Takeshita, Koki and Sasaki, Tatsuya},
  title     = {Bridging the Sim-to-Real Gap in Semiconductor Visual Program
               Synthesis via Input Binarization},
  booktitle = {Proceedings of the 34th European Signal Processing Conference (EUSIPCO)},
  year      = {2026},
  publisher = {EURASIP},
  note      = {Accepted; final citation/DOI to be updated upon publication}
}

License

MIT for both the code and these weights. The base model Qwen3-VL-8B-Instruct is subject to its own license; please review it.

Contact

Yusuke Ohtsubo — yusuke.ohtsubo.nb@hitachi.com

qwen3-vl-8b-netdsl-l2

README

Model description

Intended use

Evaluation (MIIC, 1034 real SEM test images)

Usage

Reproduce the paper's figures

NetDSL-L2

Limitations

Citation

License

Contact

Explore FriendliAI today

README

Model description

Intended use

Evaluation (MIIC, 1034 real SEM test images)

Usage

Reproduce the paper's figures

NetDSL-L2

Limitations

Citation

License

Contact