GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ngqtrung
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ngqtrung
Model Tree
Qwen/Qwen3-VL-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Offline full-set eval (VideoMME-v1 2700 + PerceptionComp 1108 + Video-Holmes 1837 = 5645 rows), scored with the repo's vero compute_score (the exact training val metric). mean = macro-mean of the 3 bench accuracies.
Keeper = global_step_80 (peak):
| metric | mean | videomme | holmes | perceptioncomp |
|---|---|---|---|---|
| base RL @80 (keeper) | 0.4918 | 0.6581 | 0.4741 | 0.3430 |
| stock Qwen3-VL-8B ckpt-0 (zero-shot) | 0.4444 | 0.6426 | 0.4143 | 0.2762 |
Trajectory (mean): 0.430 @20 → 0.475 @40 → 0.479 @60 → 0.492 @80 → then oscillates tightly 0.475–0.485 around ~0.484 through step 240 (flat plateau, no upward trend, no collapse). RL's gain concentrates where the base is weakest (pcomp 0.276→0.343); videomme is already strong at base and barely moves.
Qwen/Qwen3-VL-8B-Instruct (cold-start RL, resume_mode=auto).ngquangtrung57/verl@videorl-mods. Fully-async GRPO: FSDP2 trainer + vLLM rollouter, partial rollout.score = 0.8·accuracy + 0.2·format (FORMAT_WEIGHT=0.2, FORMAT_MIN_THINK_CHARS=100). No KL penalty.Project verl_fully_async (entity quangtrung5705-nanyang-technological-university-singapore). Main segment (train metrics only — video val is offline, not on W&B):
https://wandb.ai/quangtrung5705-nanyang-technological-university-singapore/verl_fully_async/runs/mwuxsj29
Research checkpoint — the best video keeper of an 8B controlled study, but note the headline finding: 8B video-MC RL is a documented 7-way dead-heat at ~0.485 full-set val. No lever tested (SFT warm-start, cold-base RL, exploration at several τ, larger GRPO groups) beats this ceiling; the bottleneck is outside the exploration/warm-start/RL-duration space (data / reward / task design / model scale). RL does buy a real +4-5 pt over the zero-shot base. Multiple-choice video QA, <think>…</think> then-answer format. No safety/RLHF alignment beyond the base.
python
from transformers import AutoModelForImageTextToText, AutoProcessor model_id = "ngqtrung/video-8b-grpo-base"model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")processor = AutoProcessor.from_pretrained(model_id) messages = [{ "role": "user", "content": [ {"type": "video", "video": "clip.mp4"}, {"type": "text", "text": "Answer the multiple-choice question. Reason inside <think>...</think>, then give the final letter."}, ],}]inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=1024)print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])ngquangtrung57/verl@videorl-mods; fully-async GRPO (FSDP2 + vLLM).docs/experiments_summary_8b.md).ppo_mini_batch_size=16 × require_batches=4 × rollout.n=8 = 512 trajectories/step.GROUP_VIDEO_TRAIN_MC_24F100K (5 video-MC parquets, 105,993 prompts/epoch, 24 frames / 100k pixels).1e-6, warmup 25 steps; total_epochs=2; clip_ratio 0.2 / 0.3 (clip_c=10.0); max_prompt_length=17408, max_response_length=16384; enforce_eager=true; gpu_memory_utilization=0.75; staleness 0.4.test_freq=10000); all video val is offline full-set eval on a dedicated 8×H100 node (vLLM TP1, data-parallel), every 20 fit-steps.