What this checkpoint is
Table | |
|---|
| Base model | Qwen/Qwen3.5-27B |
| Training corpus | NiuNiu0110/RST-SFT-Qwen3.5-27B |
| Checkpoint | global_step_82 |
| Architecture | Qwen3_5ForConditionalGeneration |
| Weights | 51.7 GB, bfloat16, 1199 tensors |
| Vision tower | 333 tensors, copied verbatim from the base model |
Trained with verl + FSDP2 on a pre-tokenized corpus: the Qwen3.5 loss mask is baked into
the data once (scripts/15_export_pretokenized.py) rather than recomputed per backend,
because tokenizing turns separately and concatenating them does not reproduce the
whole-conversation render for this template.
Benchmarks
None. No benchmark has been run against this checkpoint. The eval harness exists in
the repo but has not been executed on these weights, so there is no score here to quote —
not a low one, not a high one. Treat it as an untested artifact of a training run.
How it was exported
verl writes FSDP shards, which from_pretrained cannot load. These weights came from
scripts/08_prepare_eval_ckpt.sh, which merges every shard, splices the vision tower back
in from the base model (training carries only the text stack), and then verifies that the
text weights actually moved — a merge over a missing shard produces a loadable model that
is silently part-untrained, and a merge that reproduced the base is indistinguishable from
a successful one without that check.
Usage
from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained("NiuNiu0110/rst-qwen3.5-27b-sft", dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("NiuNiu0110/rst-qwen3.5-27b-sft")
License
Apache-2.0, inherited from the base model.