GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
happyhappy-jun
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
happyhappy-jun
Model Tree
Qwen/Qwen3-VL-4B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This checkpoint supersedes
happyhappy-jun/qwen3-vl-4b-uvd-racer-hl6k,
which is the same recipe, base model and step count trained on the provisional
UVD target16 subtask labels (gpu26 job 13433). Only the subtask-label source
changed. Prefer this repository (and its cs16 counterpart below) for
anything new.
hl_cu in 20260913_target_sub_a10020260913_target_sub_a100/hl_cu/checkpoint-6000 (6,000 optimizer updates)14240 on the lab AWS cluster gpu26 (us-west-2), 4 x A100 on a single node (partition a100, QoS normal8, node a100-st-p4d-cb-1 excluded)Qwen/Qwen3-VL-4B-Instruct at revision
ebb281ec70b05090aa6165b016eac8ec08e71b17 (license apache-2.0)0dd5ca0b0bc024c894af5913ae160009a215da5af212a13a63a3c79c07afe528The run's own COMPLETE.json names checkpoint-6000
({"global_step": 6000, "microbatch": 8, "effective_batch": 128, "zero_stage": 2, "source_job": "14240"}),
trainer_state.json has global_step 6000 == max_steps 6000, and
best_model_checkpoint is null (no load-best-at-end) — so checkpoint-6000 is
the final model by construction. The published files are copied byte-for-byte from
that step. No weights, tokenizer entries or configuration values were changed for
this upload; see publication_manifest.json for per-file sizes, SHA-256 values and
origins.
| Repo | Role |
|---|---|
qwen3-vl-4b-uvd-racer-hl6k-cs16-final | the same HL recipe and final labels on the 16 composite-seen (CS16) target tasks (job 14078) |
qwen3-vl-4b-uvd-racer-hl6k | superseded predecessor, provisional labels (job 13433) |
gr00t-n1.6-uvd-racer-mt80k-mix50 | low-level mid-training checkpoint of the same campaign (job 12916) |
Input — exactly one image, no frame history. A single
768 x 256 x 3 uint8 RGB tile, the observation.images.robot0_3view_concat
stream: three 256 px-wide columns, upright and unscaled, in this order —
markdown
[ agentview_left | agentview_right | eye_in_hand ]
Prompt. The user turn is rendered from this template verbatim (template
SHA-256 02507401ec656857916b1b825605a0f51bb2ce7ce71a11ef8c18a156af4e74d1,
exactly one <image> placeholder):
text
Task: {task_instruction}You are observing the robot's current camera view (three views horizontally tiled: left, right, and wrist-mounted).<image>What subtask should the robot execute right now? Reply with just the subtask description.
At training time {task_instruction} is the episode's full-task instruction
(the dataset's tasks[0]), and the <image> placeholder is expanded in place into
the tile's vision tokens. Serve it the same way: one user turn, one image at that
position.
Output. One bare subtask sentence, decoded greedily
(do_sample=False, num_beams=1, max_new_tokens=64). No chain of thought, no
formatting, no list.
The training recipe passes --min_pixels 50176 --max_pixels 115200, but under the
pinned Transformers 5.6.1 those two arguments are inert: the image
processor stores its bounds in a size dataclass, so the fork's
update_processor_pixels mutates nothing. The saved processor therefore carries the
base bounds {"longest_edge": 16777216, "shortest_edge": 65536}, and the
768 x 256 tile is fed unresized: image_grid_thw [1, 16, 48] = 768 patches =
192 image tokens after the 2 x 2 merge.
This is measured, not inferred, and gated in the eval harness — see
experiments/eval_manager/HL_INPUT_CONTRACT.md section 4 in the publishing lab's
repository. The processor_config.json shipped here (SHA-256
479b624a5b02852c3947d0b88e2570bb10464d583ad45092e2028099614bd820) is exactly that
base-bounds file, so loading the processor from this repository reproduces the
trained resolution.
Do not substitute a processor config carrying the recipe's 115200 / 50176
bounds: it shrinks the tile to 576 x 192, image_grid_thw [1, 12, 36], 108
image tokens — a resolution this model never saw.
| Item | Value |
|---|---|
| Trainer | hl_entry.py driving the pinned qwen-vl-finetune fork (runtime pin recorded in the run's launch_contract.json) |
| Data | 16 RoboCasa365 composite-unseen (CU16) target tasks, final UVD-RACER target annotations (staged dataset rc365_target_cu16_uvd_final_20260913), --dataset_use uvd_hivla_cu --data_flatten True |
| HL SFT population | 8,104 episodes → 3,202,087 train samples / 156,572 val samples |
| Steps | 6,000 (--max_steps 6000, with --num_train_epochs 15 capped by it) — 6,000 x 128 = 768,000 samples ≈ of the train split ( epoch ) |
subtask_uvd_final, sentence column
annotation.human.subtask).every_k: train stride 2, offset 0; val stride 40,
offset 1; an edge buffer of 3 frames is dropped on each side of a segment
boundary; tiles are encoded as JPEG quality 90.model.safetensors (single unsharded bf16 weight file), config.json,
generation_config.json, tokenizer.json, tokenizer_config.json,
chat_template.jinja, processor_config.json, plus this card and
publication_manifest.json.
This repository is self-sufficient for inference — model, tokenizer and processor all load from it directly:
python
from transformers import AutoProcessor, AutoModelForImageTextToTextREPO = "happyhappy-jun/qwen3-vl-4b-uvd-racer-hl6k-cu16-final"processor = AutoProcessor.from_pretrained(REPO)model = AutoModelForImageTextToText.from_pretrained(REPO, dtype="auto", device_map="auto")# greedy decoding, as trained and evaluated:# model.generate(**inputs, do_sample=False, num_beams=1, max_new_tokens=64)
vocab.json / merges.txt are not shipped and are not needed: tokenizer.json
is self-contained, so the fast tokenizer loads without them.
The run wrote two exports at step 6,000: the checkpoint directory
20260913_target_sub_a100/hl_cu/checkpoint-6000, which the run's own completion marker
designates, and a final save_model export at the run root. The six
weight/config/tokenizer files here are copied byte-for-byte from
checkpoint-6000. The pinned trainer passes only the tokenizer to the
Trainer as its processing class, so no checkpoint-N/ directory contains a
processor config; processor_config.json is therefore copied byte-for-byte from
the run-root save_model export — the same step of the same job — purely so that
AutoProcessor.from_pretrained resolves against this repository. It is a
Qwen3VLProcessor config holding the Qwen2VLImageProcessor settings (patch size
16, merge size 2, temporal patch size 2, mean/std 0.5) and the video-processor
settings (fps 2, frame sampling on); on this Transformers version that single file
replaces the base repo's separate preprocessor_config.json and
video_preprocessor_config.json. No preprocessing setting was changed relative
to the base model. Per-file sizes and SHA-256 values, each tagged with its origin,
are in .
Not included, by design: DeepSpeed ZeRO optimizer shards (global_step6000/),
optimizer/scheduler/RNG state, trainer_state.json, training_args.bin, training
logs and the training data. This is a model export for inference, not a
complete optimizer-state resume bundle.
python
from huggingface_hub import snapshot_downloadcheckpoint_dir = snapshot_download("happyhappy-jun/qwen3-vl-4b-uvd-racer-hl6k-cu16-final",# revision=<commit sha of the single publication commit>)
The whole payload is submitted in one atomic commit on a freshly created
repository, so that commit's sha is the only model revision here. It cannot be
written inside this file (the text would change the commit it names); the pinned
snapshot_download(..., revision=...) snippet is recorded in the publishing lab's
publication log alongside the per-file SHA-256 values.
No GPU inference or robot-rollout evaluation was run for this upload. Upload verification covers source checksum identity, the Safetensors header and contiguous tensor-offset coverage of the single weight file, payload content scanning, and Hub file integrity after the commit — nothing more. No subtask-accuracy, task-success, hierarchical-policy or safety-performance number is claimed by this model card, and none should be inferred from the training losses above: they are training/validation cross-entropy on the sampled frames of this split, not a behavioural metric.
Further scope limits worth stating plainly: the model was trained on 16 target tasks of one simulated benchmark with a fixed three-view tile, emits language only (it drives no actuator by itself), and was optimized for composite-unseen (CU16) RoboCasa365 tasks — behaviour on other task distributions, camera layouts or tile orders is untested.
Inherits the base model's Apache-2.0 license. Attribution: Qwen team, Alibaba
Cloud — Qwen/Qwen3-VL-4B-Instruct. This repository is an independent derivative
fine-tune and is not endorsed by the Qwen team.
Outputs are generated language and are not guaranteed to be correct or safe. Validate in simulation and use appropriate robot safety controls before any physical deployment.
| longer-schedule low-level pretraining/mid-training sibling (job 11798) |
gr00t-n1.6-uvd-racer-ft-cs60k-fulltask | full-task-language low-level FT comparator, composite-seen split |
gr00t-n1.6-uvd-racer-ft-at60k-fulltask | full-task-language low-level FT comparator, atomic-seen split |
trainer_state.json0.23984170447504646| Effective batch | 128 (per-device 8 x gradient accumulation 4 x 4 GPUs) |
| Learning rate | 6e-5, cosine schedule, warmup ratio 0.05 (5%); step-6,000 LR 4.5566039474920216e-12, i.e. the schedule ran to its end |
| Weight decay | 0; --max_grad_norm 1 |
| Optimizer | adamw_torch, bf16 |
| Trainable | LLM only (--tune_mm_llm True); vision tower and multimodal merger frozen (--tune_mm_vision False --tune_mm_mlp False) |
| Sequence | --model_max_length 8192, gradient checkpointing on, --dataloader_num_workers 8 |
| Seed | 42 |
| Eval / save | --eval_strategy steps --eval_steps 1000 --save_strategy steps --save_steps 1000 --save_total_limit 2 --logging_steps 1 |
| Final losses | train 0.058870457112789154, eval 0.0566091351211071 (step 6,000) |
| Runtime | DeepSpeed 0.19.3, ZeRO stage 2, Transformers 5.6.1 |
publication_manifest.json