samuellimabraz

Qwen3-VL-8B-rslora-r128

README

License: apache-2.0

Model Description

This model is a fine-tuned version of Qwen3-VL-8B-Instruct specialized for quantum computing tasks using Qiskit 2.0. This model can interpret visual representations of quantum computing: circuit diagrams, Bloch spheres, and measurement histograms.

The model was trained using Rank-Stabilized Low-Rank Adaptation (rsLoRA) with rank 128 for 3 epochs on the Quantum Assistant Dataset. Note: This is an experimental model that showed severe overfitting during training (Eval Loss increased to 0.789), and was not evaluated on external benchmarks. It is included in the collection for research purposes.

Key Capabilities

Code Generation: Generate complete Qiskit code from natural language descriptions
Function Completion: Complete function bodies from signatures and docstrings
Visual Understanding: Interpret quantum circuit diagrams, Bloch spheres, and histograms
Conceptual Explanations: Answer questions about quantum computing theory
Qiskit 2.0 Compliant: Uses modern APIs (SamplerV2, EstimatorV2, generate_preset_pass_manager)

Training Evaluation

This model was part of the Phase 2 hyperparameter optimization experiments. It was not evaluated on external benchmarks (Qiskit HumanEval, Qiskit HumanEval Hard, or synthetic test set) due to severe overfitting observed during training.

Internal Validation Metrics

Table with columns: Metric, Value, Step
Metric	Value	Step
Eval Loss	0.789	276 (final)
Eval Token Accuracy	0.822	276 (final)
Train Loss	0.058	276 (final)
Train Token Accuracy	0.982	276 (final)
Training Runtime	2,754 seconds (~45.9 min)	3 epochs

Overfitting Analysis

This configuration showed severe overfitting:

Training loss dropped dramatically from 0.243 (step 180) to 0.058 (step 276) - a 76.1% reduction
Training accuracy increased from 0.924 to 0.982
Evaluation loss increased from 0.660 (step 180) to 0.789 (step 276) - a 19.5% degradation ⚠️
Evaluation accuracy remained stable at ~0.822

The model memorized the training data rather than learning generalizable patterns, making it unsuitable for production use.

Conclusion: This experiment demonstrates that rank 128 with 3 epochs is excessive for this dataset size (5,837 training samples), leading to memorization instead of generalization.

Training Strategy

The experimental strategy was organized in two phases: PEFT technique selection and hyperparameter optimization.

Phase 1: PEFT Variant Comparison

Five LoRA variants were compared with controlled configuration (r=16, α=32, 1 epoch):

Table with columns: Variant, Eval Loss ↓, Eval Accuracy ↑, Runtime (s)
Variant	Eval Loss ↓	Eval Accuracy ↑	Runtime (s)
rsLoRA	0.622	0.818	1,060
DoRA	0.622	0.818	2,307
rsLoRA (frozen aligner)	0.623	0.817	1,057
LoRA (vanilla)	0.646	0.812

Key findings:

rsLoRA and DoRA achieved equivalent performance (Eval Loss 0.622)
DoRA has 2.18× computational overhead (2,307s vs 1,060s) due to magnitude-direction decomposition
rsLoRA selected for optimal performance-efficiency trade-off

Phase 2: Rank and Epoch Optimization

With rsLoRA selected, the impact of adapter rank and training duration was investigated:

Table with columns: Configuration, Eval Loss ↓, Eval Accuracy ↑, Notes
Configuration	Eval Loss ↓	Eval Accuracy ↑	Notes
r=32, 1 epoch	0.607	0.821	Optimal trade-off
r=64, 1 epoch	0.609	0.822	Marginal improvement
r=16, 1 epoch	0.622	0.818	Baseline rsLoRA
r=32, 2 epochs	0.638	0.825	Slight overfitting

Conclusions: rsLoRA with r=32 and 1-2 epochs maximizes generalization while avoiding memorization of the synthetic dataset.

Model Collection

This model is part of the Quantum Assistant collection. All models are merged versions ready for inference:

Table with columns: Model, Configuration, Description
Model	Configuration	Description
Qwen3-VL-8B-rslora-r32-2	rsLoRA r=32, 2 epochs	Best overall performance
Qwen3-VL-8B-rslora-r32	rsLoRA r=32, 1 epoch	Best generalization
Qwen3-VL-8B-rslora-r64	rsLoRA r=64, 1 epoch	Higher capacity

Usage

With vLLM

bash
python -m vllm.entrypoints.openai.api_server \
    --host 0.0.0.0 \
    --port 8000 \
    --model samuellimabraz/Qwen3-VL-8B-rslora-r128 \
    --gpu-memory-utilization 0.92 \
    --max-model-len 12288 \
    --max-num-seqs 16 \
    --max-num-batched-tokens 49152 \
    --enable-chunked-prefill \
    --enable-prefix-caching

With Transformers

python
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "samuellimabraz/Qwen3-VL-8B-rslora-r128",
    torch_dtype="auto",
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("samuellimabraz/Qwen3-VL-8B-rslora-r128")

messages = [
    {"role": "system", "content": "You are a quantum computing expert assistant specializing in Qiskit."},
    {"role": "user", "content": "Create a function that builds a 3-qubit GHZ state and returns the circuit."}
]

messages_with_image = [
    {"role": "system", "content": "You are a quantum computing expert assistant specializing in Qiskit."},
    {"role": "user", "content": [
        {"type": "image", "image": "path/to/circuit.png"},
        {"type": "text", "text": "Implement the quantum circuit shown in the image."}
    ]}
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt"
).to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=1024)
output = processor.batch_decode(
    generated_ids[:, inputs.input_ids.shape[1]:], 
    skip_special_tokens=True
)[0]
print(output)

Training Details

Dataset

Training Data: Quantum Assistant Dataset
Train Samples: 5,837 (45.1% multimodal)
Validation Samples: 1,239 (45.2% multimodal)
Task Distribution: 30% function completion, 32% code generation, 38% QA
Categories: 7 quantum computing domains

Training Configuration

Table with columns: Parameter, Value
Parameter	Value
Base Model	Qwen/Qwen3-VL-8B-Instruct
PEFT Method	rsLoRA (Rank-Stabilized LoRA)
Rank (r)	128
Alpha (α)	256
Dropout	0.10
Target Modules	all-linear
Learning Rate	1e-4
LR Scheduler	Cosine

Freezing Strategy

Table with columns: Component, Status
Component	Status
Vision Encoder (ViT)	❄️ Frozen
Vision-Language Aligner	🔥 Trainable
Language Model (LLM)	🔥 Trainable

Training Infrastructure

GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition (96GB VRAM)
Training Time: ~45.9 minutes (3 epochs)
Tracking: Weights & Biases | TensorBoard

System Prompt

markdown
You are a quantum computing expert assistant specializing in Qiskit.
Provide accurate, clear, and well-structured responses about quantum computing concepts,
algorithms, and code implementation. Use Qiskit 2.0 best practices.

Intended Uses & Limitations

Intended Uses

Educational assistance: Learning quantum computing concepts with Qiskit
Code generation: Creating Qiskit circuits from descriptions or diagrams
Documentation: Understanding quantum circuit visualizations
Research prototyping: Rapid development of quantum algorithms

Limitations

Domain specificity: Optimized for Qiskit 2.0; may generate deprecated APIs for older versions
Dataset size: Trained on 5,837 samples; may underperform on rare edge cases
Category imbalance: Better performance on circuits_and_gates than primitives_and_execution
Hardware specifics: Limited coverage of IBM Quantum hardware-specific optimizations
Execution: Generated code requires verification before running on real quantum hardware

Bias and Risks

Model may perpetuate patterns from training data
Visual understanding limited to common diagram styles in Qiskit documentation
May generate syntactically correct but logically incorrect quantum algorithms
Should not be used for production quantum computing without human review

Citation

If you use this model in your research, please cite:

bibtex
@article{braz2026quantumassistant,
  title     = {Quantum Assistant: Specialization of Multimodal Vision-Language Models for Quantum Computing},
  author    = {Braz, Samuel Lima and Leite, Jo{\~a}o Paulo Reus Rodrigues},
  journal   = {Expert Systems with Applications},
  year      = {2026},
  issn      = {0957-4174},
  doi       = {10.1016/j.eswa.2026.133931},
  url       = {https://doi.org/10.1016/j.eswa.2026.133931},
  publisher = {Elsevier}
}

Paper: doi:10.1016/j.eswa.2026.133931
Dataset: samuellimabraz/quantum-assistant
Model Collection: Quantum Assistant Models
Demo: Quantum Assistant Space
Code: GitHub Repository

Acknowledgments

IBM Quantum and Qiskit team for open-source documentation
Qwen Team for the base model
UNIFEI (Universidade Federal de Itajubá) for academic support
Advisor: Prof. João Paulo Reus Rodrigues Leite

License

This model is released under the Apache 2.0 License.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

samuellimabraz

Model Tree

Base

Qwen/Qwen3-VL-8B-Instruct

Fine-tuned

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Description

Key Capabilities

Code Generation: Generate complete Qiskit code from natural language descriptions
Function Completion: Complete function bodies from signatures and docstrings
Visual Understanding: Interpret quantum circuit diagrams, Bloch spheres, and histograms
Conceptual Explanations: Answer questions about quantum computing theory
Qiskit 2.0 Compliant: Uses modern APIs (SamplerV2, EstimatorV2, generate_preset_pass_manager)

Training Evaluation

Internal Validation Metrics

Table with columns: Metric, Value, Step
Metric	Value	Step
Eval Loss	0.789	276 (final)
Eval Token Accuracy	0.822	276 (final)
Train Loss	0.058	276 (final)
Train Token Accuracy	0.982	276 (final)
Training Runtime	2,754 seconds (~45.9 min)	3 epochs

Overfitting Analysis

This configuration showed severe overfitting:

Training loss dropped dramatically from 0.243 (step 180) to 0.058 (step 276) - a 76.1% reduction
Training accuracy increased from 0.924 to 0.982
Evaluation loss increased from 0.660 (step 180) to 0.789 (step 276) - a 19.5% degradation ⚠️
Evaluation accuracy remained stable at ~0.822

The model memorized the training data rather than learning generalizable patterns, making it unsuitable for production use.

Conclusion: This experiment demonstrates that rank 128 with 3 epochs is excessive for this dataset size (5,837 training samples), leading to memorization instead of generalization.

Training Strategy

The experimental strategy was organized in two phases: PEFT technique selection and hyperparameter optimization.

Phase 1: PEFT Variant Comparison

Five LoRA variants were compared with controlled configuration (r=16, α=32, 1 epoch):

Table with columns: Variant, Eval Loss ↓, Eval Accuracy ↑, Runtime (s)
Variant	Eval Loss ↓	Eval Accuracy ↑	Runtime (s)
rsLoRA	0.622	0.818	1,060
DoRA	0.622	0.818	2,307
rsLoRA (frozen aligner)	0.623	0.817	1,057
LoRA (vanilla)	0.646	0.812

Key findings:

rsLoRA and DoRA achieved equivalent performance (Eval Loss 0.622)
DoRA has 2.18× computational overhead (2,307s vs 1,060s) due to magnitude-direction decomposition
rsLoRA selected for optimal performance-efficiency trade-off

Phase 2: Rank and Epoch Optimization

With rsLoRA selected, the impact of adapter rank and training duration was investigated:

Table with columns: Configuration, Eval Loss ↓, Eval Accuracy ↑, Notes
Configuration	Eval Loss ↓	Eval Accuracy ↑	Notes
r=32, 1 epoch	0.607	0.821	Optimal trade-off
r=64, 1 epoch	0.609	0.822	Marginal improvement
r=16, 1 epoch	0.622	0.818	Baseline rsLoRA
r=32, 2 epochs	0.638	0.825	Slight overfitting

Conclusions: rsLoRA with r=32 and 1-2 epochs maximizes generalization while avoiding memorization of the synthetic dataset.

Model Collection

This model is part of the Quantum Assistant collection. All models are merged versions ready for inference:

Table with columns: Model, Configuration, Description
Model	Configuration	Description
Qwen3-VL-8B-rslora-r32-2	rsLoRA r=32, 2 epochs	Best overall performance
Qwen3-VL-8B-rslora-r32	rsLoRA r=32, 1 epoch	Best generalization
Qwen3-VL-8B-rslora-r64	rsLoRA r=64, 1 epoch	Higher capacity

Usage

With vLLM

bash
python -m vllm.entrypoints.openai.api_server \
    --host 0.0.0.0 \
    --port 8000 \
    --model samuellimabraz/Qwen3-VL-8B-rslora-r128 \
    --gpu-memory-utilization 0.92 \
    --max-model-len 12288 \
    --max-num-seqs 16 \
    --max-num-batched-tokens 49152 \
    --enable-chunked-prefill \
    --enable-prefix-caching

With Transformers

python
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "samuellimabraz/Qwen3-VL-8B-rslora-r128",
    torch_dtype="auto",
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("samuellimabraz/Qwen3-VL-8B-rslora-r128")

messages = [
    {"role": "system", "content": "You are a quantum computing expert assistant specializing in Qiskit."},
    {"role": "user", "content": "Create a function that builds a 3-qubit GHZ state and returns the circuit."}
]

messages_with_image = [
    {"role": "system", "content": "You are a quantum computing expert assistant specializing in Qiskit."},
    {"role": "user", "content": [
        {"type": "image", "image": "path/to/circuit.png"},
        {"type": "text", "text": "Implement the quantum circuit shown in the image."}
    ]}
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt"
).to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=1024)
output = processor.batch_decode(
    generated_ids[:, inputs.input_ids.shape[1]:], 
    skip_special_tokens=True
)[0]
print(output)

Training Details

Dataset

Training Data: Quantum Assistant Dataset
Train Samples: 5,837 (45.1% multimodal)
Validation Samples: 1,239 (45.2% multimodal)
Task Distribution: 30% function completion, 32% code generation, 38% QA
Categories: 7 quantum computing domains

Training Configuration

Table with columns: Parameter, Value
Parameter	Value
Base Model	Qwen/Qwen3-VL-8B-Instruct
PEFT Method	rsLoRA (Rank-Stabilized LoRA)
Rank (r)	128
Alpha (α)	256
Dropout	0.10
Target Modules	all-linear
Learning Rate	1e-4
LR Scheduler	Cosine

Freezing Strategy

Table with columns: Component, Status
Component	Status
Vision Encoder (ViT)	❄️ Frozen
Vision-Language Aligner	🔥 Trainable
Language Model (LLM)	🔥 Trainable

Training Infrastructure

GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition (96GB VRAM)
Training Time: ~45.9 minutes (3 epochs)
Tracking: Weights & Biases | TensorBoard

System Prompt

markdown
You are a quantum computing expert assistant specializing in Qiskit.
Provide accurate, clear, and well-structured responses about quantum computing concepts,
algorithms, and code implementation. Use Qiskit 2.0 best practices.

Intended Uses & Limitations

Intended Uses

Educational assistance: Learning quantum computing concepts with Qiskit
Code generation: Creating Qiskit circuits from descriptions or diagrams
Documentation: Understanding quantum circuit visualizations
Research prototyping: Rapid development of quantum algorithms

Limitations

Domain specificity: Optimized for Qiskit 2.0; may generate deprecated APIs for older versions
Dataset size: Trained on 5,837 samples; may underperform on rare edge cases
Category imbalance: Better performance on circuits_and_gates than primitives_and_execution
Hardware specifics: Limited coverage of IBM Quantum hardware-specific optimizations
Execution: Generated code requires verification before running on real quantum hardware

Bias and Risks

Model may perpetuate patterns from training data
Visual understanding limited to common diagram styles in Qiskit documentation
May generate syntactically correct but logically incorrect quantum algorithms
Should not be used for production quantum computing without human review

Citation

If you use this model in your research, please cite:

bibtex
@article{braz2026quantumassistant,
  title     = {Quantum Assistant: Specialization of Multimodal Vision-Language Models for Quantum Computing},
  author    = {Braz, Samuel Lima and Leite, Jo{\~a}o Paulo Reus Rodrigues},
  journal   = {Expert Systems with Applications},
  year      = {2026},
  issn      = {0957-4174},
  doi       = {10.1016/j.eswa.2026.133931},
  url       = {https://doi.org/10.1016/j.eswa.2026.133931},
  publisher = {Elsevier}
}

Paper: doi:10.1016/j.eswa.2026.133931
Dataset: samuellimabraz/quantum-assistant
Model Collection: Quantum Assistant Models
Demo: Quantum Assistant Space
Code: GitHub Repository

Acknowledgments

IBM Quantum and Qiskit team for open-source documentation
Qwen Team for the base model
UNIFEI (Universidade Federal de Itajubá) for academic support
Advisor: Prof. João Paulo Reus Rodrigues Leite

License

This model is released under the Apache 2.0 License.

Qwen3-VL-8B-rslora-r128

README

Model Description

Key Capabilities

Training Evaluation

Internal Validation Metrics

Overfitting Analysis

Training Strategy

Phase 1: PEFT Variant Comparison

Phase 2: Rank and Epoch Optimization

Model Collection

Usage

With vLLM

With Transformers

Training Details

Dataset

Training Configuration

Freezing Strategy

Training Infrastructure

System Prompt

Intended Uses & Limitations

Intended Uses

Limitations

Bias and Risks

Citation

Related Resources

Acknowledgments

License

Explore FriendliAI today

README

Model Description

Key Capabilities

Training Evaluation

Internal Validation Metrics

Overfitting Analysis

Training Strategy

Phase 1: PEFT Variant Comparison

Phase 2: Rank and Epoch Optimization

Model Collection

Usage

With vLLM

With Transformers

Training Details

Dataset

Training Configuration

Freezing Strategy

Training Infrastructure

System Prompt

Intended Uses & Limitations

Intended Uses

Limitations

Bias and Risks

Citation

Related Resources

Acknowledgments

License