wflying

Qwen3.5-4B-RL-MATH

README

License: apache-2.0

Model details

Base model: Qwen/Qwen3.5-4B
Training dataset: wflying/math-rlvr-15k
Training method: GRPO-style reinforcement learning with verifiable mathematical rewards
Primary use: Mathematical reasoning and competition-style problem solving
Architecture: Qwen3.5 multimodal conditional-generation architecture, approximately 4.66B parameters, BF16 weights
Training mode: Non-thinking (enable_thinking=False)

Evaluation results

All results below were obtained in non-thinking mode. The table reports the evaluation scores for the released model and two Qwen3.5 baselines.

Table with columns: Model, AIME24, AIME25, HMMT25, GPQA-Diamond, zebralogicbench, LiveCodeBench-v6
Model	AIME24	AIME25	HMMT25	GPQA-Diamond	zebralogicbench	LiveCodeBench-v6
Qwen3.5-35B-A3B	64.79	60	48.75	82.83	90.6	71.85
Qwen3.5-4B	58.33	38.33	30	71.21	82.6	52.13
Qwen3.5-4B-RL-MATH	74.38	58.54	50.42	76.89	89.8	62.65

Under this evaluation setup, Qwen3.5-4B-RL-MATH improves over the Qwen3.5-4B baseline by 16.05 points on AIME24, 20.21 on AIME25, 20.42 on HMMT25, 5.68 on GPQA-Diamond, 7.20 on zebralogicbench, and 10.52 on LiveCodeBench-v6.

Usage example

Use a recent version of transformers with Qwen3.5 support. The example below follows the non-thinking configuration used for evaluation.

python
import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "wflying/Qwen3.5-4B-RL-MATH"

model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Solve the problem and put the final answer in \\boxed{}: What is the sum of the first 20 positive integers?",
            }
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    generated_ids = model.generate(**inputs, max_new_tokens=512)

generated_ids = generated_ids[:, inputs.input_ids.shape[1]:]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

Limitations

This checkpoint is specialized for mathematical reasoning; performance may vary on unrelated tasks.
The reported scores use non-thinking mode and may not transfer directly to other prompts, decoding settings, or evaluation implementations.
The RLVR stage used text-only mathematical data. The model retains the multimodal architecture of its base model, but multimodal behavior was not separately optimized or evaluated in this post-training stage.
Generated answers can still be incorrect. Verify outputs before using them in high-stakes settings.

License

This model inherits the Apache License 2.0 from the base model. Users must also comply with the applicable terms of the training dataset and dependent components.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

wflying

Model Tree

Base

Qwen/Qwen3.5-4B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality