GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ut-amrl
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ut-amrl
Model Tree
Qwen/Qwen3-VL-2B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Given a natural-language goal and a short history of RGB observations, the model proposes an image-space trajectory and critiques its own proposal. Motion planning and critique were co-trained into this single checkpoint, so both roles are served by one vLLM engine at deployment time.
Converting the accepted plan into metric BEV waypoints is handled by a separate grounding policy, which is not part of this repository.
The deployment stack loads this model through vLLM by repo ID. In
legged_deployment/config/vllm_server_sft.yaml:
yaml
model_overrides: vlm: provider_kwargs: model: "ut-amrl/foresight-qwen3vl-2b-sft"It also loads directly with transformers:
python
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration model = Qwen3VLForConditionalGeneration.from_pretrained( "ut-amrl/foresight-qwen3vl-2b-sft", dtype="bfloat16", device_map="auto")processor = AutoProcessor.from_pretrained("ut-amrl/foresight-qwen3vl-2b-sft")Prompts are constructed by the Foresight codebase; see https://github.com/ut-amrl/foresight_public for the runnable deployment stack.
Fine-tuned from Qwen/Qwen3-VL-2B-Instruct with LoRA (rank 64, alpha 64) applied to
all linear projections in both the language and vision towers. The adapter has been
merged into the base weights, so no PEFT dependency is needed at inference.
Weights are bfloat16.
bibtex
@article{zhang2026foresight, title={Foresight: Iterative Reasoning About Clues that Matter for Navigation}, author={Zhang, Arthur and Qi, Carl and Su, Donne and Meng, Xiangyun and Zhang, Amy and Biswas, Joydeep}, journal={arXiv preprint arXiv:2606.12550}, year={2026}}@article{zhang2026foresight, title={Foresight: Iterative Reasoning About Clues that Matter for Navigation}, author={Zhang, Arthur and Qi, Carl and Su, Donne and Meng, Xiangyun and Zhang, Amy and Biswas, Joydeep}, journal={arXiv preprint arXiv:2606.12550}, year={2026}}