GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ambient-intelligence-labs
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ambient-intelligence-labs
Model Tree
Qwen/Qwen3.5-2B
Input Modalities
Output Modalities
Supported Functionality
Given ~100 uniformly sampled frames spanning a ten-minute egocentric video, plus a four-option question, the model answers in one greedy forward pass — no retrieval, no tool calls, no ensemble. It emits a timestamped description, then a structured answer:
markdown
<video_description>…</video_description><answer><choice>C</choice><reason>…</reason><citation>…</citation></answer>
This is the perception step of a tool-using agentic pipeline, distilled out of it. The agent's senior orchestrator — which retrieves clips and revises over multiple turns — is discarded, not imitated: its value is the multi-turn loop, which no single forward pass can express. Collapsing the pipeline this way takes per-sample latency from ~393 s to ~32 s.
python
import torch, jsonfrom transformers import AutoModelForImageTextToText, AutoProcessorfrom peft import PeftModelfrom PIL import ImageBASE = "Qwen/Qwen3.5-2B"proc = AutoProcessor.from_pretrained(BASE)model = PeftModel.from_pretrained(AutoModelForImageTextToText.from_pretrained(BASE, dtype=torch.bfloat16, device_map="cuda"),"infinitylogesh/egolongqa-2b-distill-adapter").eval()query = ("Provide the detailed video description and answer to the question now: "f"Question:{question} {mcq_options}")msgs = [{"role": "system", "content": [{"type": "text", "text": SYSTEM_PROMPT}]},{"role": "user", "content": [{"type": "image"} for _ in frames]+ [{"type": "text", "text": query}]}]text = proc.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)batch = proc(text=[text], images=[Image.open(p).convert("RGB") for p in frames],return_tensors="pt", images_kwargs={"max_pixels": 331776, "min_pixels": 3136}).to(model.device)out = model.generate(**batch, max_new_tokens=8192, do_sample=False)
Three settings are not free parameters:
max_pixels=331776 at inference, though the adapter was trained at 50176. Training cheap and
inferring high is worth +5.8 points; the adapter learns the task, not the resolution.max_new_tokens=8192. At the common default of 768, 3–9 % of completions are cut
mid-description, never emit <answer>, and score wrong regardless of what the model concluded.SYSTEM_PROMPT is prompts/junior_system.txt
in the code repo — 2000 bytes, md5 f342eb86cceb9ee18c9fba0ceae9f013. It carries two trailing
spaces; a copy with them stripped changed 10 of 70 held-out answers.
Frames must be extracted the same way the traces were, or the model sees different moments than it was trained on:
bash
ffmpeg -i VIDEO -vf "fps=min(2,100/DURATION),scale=768:768:force_original_aspect_ratio=decrease" \-q:v 3 frames/%03d.jpg # take the first 100
Merged, this is 2.2132 B — over the divisional limit, and the embedding table is 23 % of it (248,320 rows × 2048, weight-tied). Pruning the vocabulary to 143,469 rows yields 1.9985 B with a measured logit difference of exactly 0 on retained rows and byte-identical generations.
The prune is reproducible from this adapter: merging and pruning with the keep-set in the code repo
regenerates the submitted model.safetensors bit-for-bit (md5 896c434c402450f57315daee869bc68e).
The output side is not lossless. Input-side pruning has a byte fallback; the output side has none — if the argmax lands on a removed row the model emits a different word. Capture the keep-set on the GPU that will serve the model.
Distilled from 2,936 teacher traces over 605 videos (median 5 distinct descriptions per
question), harvested from agentic runs and filtered to keep only passes whose <choice> matched
ground truth — so the student imitates teacher parity, not teacher average. All teachers are
open-weight (gemma-4-31b-it, qwen3.5-122b-a10b, Qwen3.6-27B/35B).
LoRA r=16 α=32, 1 epoch (epoch 2 was worse in all five runs), cosine schedule, batch size 1 × 8
accumulation, 3× token weight on the <choice> span, max_pixels=50176, bf16.
infinitylogesh/egolongqa-synth-annotationsbibtex
@techreport{umapathi2026egolongqa,title = {Ambient @ EgoLongQA 2026: Distilling Perception, Not Orchestration,into a Sub-2B Model},author = {Umapathi, Logesh Kumar},year = {2026},institution = {Team Ambient},note = {Wearable AI Challenge @ ECCV 2026, EgoLongQA track}}