jackcai1206

qwen3-4b-longcot-sft-vanilla

README

License: apache-2.0

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer REPO = "jackcai1206/qwen3-4b-longcot-sft-vanilla"model = AutoModelForCausalLM.from_pretrained(REPO, dtype="bfloat16", device_map="auto")tok   = AutoTokenizer.from_pretrained(REPO)

Precision

Training kept fp32 master weights (16.1 GB). Downstream use constructs the model in bf16, so those weights are truncated to bf16 on load — this export is bf16 and therefore bit-identical to what training and inference actually consume, at half the size. Verified tensor by tensor against the fp32 checkpoint. lm_head is tied to embed_tokens in the source checkpoint (same storage), so it is reconstructed by from_pretrained rather than stored separately.

Evaluation

MATH-500 (strict): 87.4.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

jackcai1206

Model Tree

Base

Qwen/Qwen3-4B-Base

Fine-tuned

this model

Input Modalities

Text

Output Modalities