GaviZhou

qwen3-vl-8b-i2c-hallu-v6-32k

README

License: apache-2.0

Intended use

Use this model for research and development workflows that compare a rendered or target figure with candidate code. The desired response format is:

json
{
  "errors": [
    {
      "category": "error category",
      "description": "description of the discrepancy",
      "code_span": {
        "start_line": 12,
        "end_line": 14,
        "anchor": "relevant code fragment"
      }
    }
  ]
}

code_span is optional. Line numbers are 1-based and should refer to the candidate code supplied in the prompt.

Training

Base model: Qwen/Qwen3-VL-8B-Instruct
Fine-tuning method: full-parameter SFT with ms-swift (tuner_type=full)
Precision: BF16
Context length: 32,768 tokens
Packing: bin packing to 32,768 tokens
Training data: 7,014 examples
Validation data: 373 examples
Data split: grouped by ground-truth image; ground-truth-image and candidate-code overlap between train and validation splits are both zero
Epochs / optimizer steps: 2 / 198
Effective batch size: 8 (2 devices × batch size 1 × gradient accumulation 4)
Optimizer settings: AdamW, learning rate 1e-5, weight decay 0.01
Learning-rate schedule: cosine decay with 5% warmup
Loss: assistant-response loss (loss_scale=last_round)

The checkpoint published here is the best checkpoint from the first training run: checkpoint-198.

Validation results

Teacher-forced validation metrics at checkpoint-198:

Table with columns: Metric, Value
Metric	Value
`eval_loss`	0.352678
`eval_token_acc`	0.900092

The packed validation set contains 42 sequences. These are likelihood and token-level metrics, not direct measurements of error-detection quality or hallucination rate. For model selection or production use, additionally evaluate generated answers on held-out examples with JSON/schema validity, error-category precision/recall/F1, and code-anchor/line-alignment metrics.

Quick start

Install a recent Transformers release with Qwen3-VL support, then load the model as an image-text-to-text model:

python
import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "GaviZhou/qwen3-vl-8b-i2c-hallu-v6-32k"

model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

candidate_code = """# Candidate plotting code; retain 1-based line references."""
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "/path/to/ground_truth.png"},
            {
                "type": "text",
                "text": (
                    "Compare the ground-truth image with the candidate code. "
                    "Return only the required JSON object.\n\n"
                    f"Candidate code:\n{candidate_code}"
                ),
            },
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=1024)
generated_ids = [
    output_ids[len(input_ids):]
    for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

The upstream Qwen3-VL model card contains additional examples and deployment guidance.

Limitations and responsible use

This model can produce incorrect, incomplete, or invalid JSON output. Parse and validate its output before using it in an automated workflow.
The reported validation metrics are not a substitute for task-level generation evaluation or human review.
It is not a general-purpose code verifier, security analyzer, or guarantee that a figure faithfully represents code.
Users are responsible for ensuring that their input images, code, and any downstream use comply with applicable licenses, privacy requirements, and policies.

License

This checkpoint is derived from Qwen/Qwen3-VL-8B-Instruct, which is released under the Apache-2.0 license. Users must also ensure that their use of the fine-tuning data and downstream inputs is permitted.

Citation

If you use this model, please also cite the base model:

bibtex
@misc{qwen3technicalreport,
  title={Qwen3 Technical Report},
  author={Qwen Team},
  year={2025},
  eprint={2505.09388},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2505.09388}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

GaviZhou

Model Tree

Base

Qwen/Qwen3-VL-8B-Instruct

Fine-tuned

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer