GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Alibaba-DAMO-Academy
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Alibaba-DAMO-Academy
Model Tree
Qwen/Qwen3.5-122B-A10B
Input Modalities
Output Modalities
Supported Functionality
We present RynnBrain 1.1, a systematic upgrade of RynnBrain for embodied intelligence. RynnBrain 1.1 is released in three scales: 2B, 9B, and 122B-A10B, extending the model family from compact dense models to its first 122B-level sparse-MoE model.
| Model | Base Model | HuggingFace | ModelScope |
|---|---|---|---|
| RynnBrain1.1-2B | Qwen3.5-2B | Link | Link |
| RynnBrain1.1-9B | Qwen3.5-9B | Link | Link |
Minimal dependencies
shell
pip install transformers==5.2.0Run text generation
python
import torchfrom transformers import AutoModelForImageTextToText, AutoProcessor conversation = [ { "role": "user", "content": [ {"type": "image", "image": "cookbooks/assets/object_location/images/000000086408.jpg"}, { "type": "text", "text": "What appliance can be used to heat food quickly.\nGenerate coordinates for one object bounding box. Constraints: x1,y1,x2,y2 in [0,1000]. Response must be in the format: ", }, ], }] model_path = "Alibaba-DAMO-Academy/RynnBrain1.1-2B"processor = AutoProcessor.from_pretrained(model_path) model = AutoModelForImageTextToText.from_pretrained( model_path, dtype=torch.bfloat16,)model.to("cuda") model_inputs = processor.apply_chat_template( conversation, add_generation_prompt=True, enable_thinking=False, tokenize=True, return_dict=True, return_tensors="pt",)model_inputs = model_inputs.to("cuda") output_ids = model.generate( **model_inputs, max_new_tokens=256, do_sample=False,)output_ids = output_ids[:, model_inputs["input_ids"].size(1) :]response = processor.decode(output_ids[0], skip_special_tokens=True)print(response)For installation and advanced usages, please refer to the official documentation.
OpenAI-Compatible Serving
shell
# launch serverpython3 -m sglang.launch_server --model-path Alibaba-DAMO-Academy/RynnBrain1.1-2B --host 0.0.0.0 --port 8000python
# inference using openai apiimport base64import io from openai import OpenAIfrom PIL import Image def pil_to_url(image: Image.Image): image_format = image.format if image.format else 'PNG' buffered = io.BytesIO() image.save(buffered, format=image_format) img_str = base64.b64encode(buffered.getvalue()).decode('utf-8') return f'data:image/{image_format.lower()};base64,{img_str}' messages = [ { 'role': 'user', 'content': [ {'type': 'image_url', 'image_url': {'url': pil_to_url(Image.open('cookbooks/assets/object_location/images/000000086408.jpg'))}}, {'type': 'text', 'text': 'What appliance can be used to heat food quickly.\nGenerate coordinates for one object bounding box. Constraints: x1,y1,x2,y2 ∈ [0,1000]. Response must be in the format: '}, ], }] client = OpenAI(api_key="", base_url="http://localhost:8000/v1")response = client.chat.completions.create( model="default", messages=messages, stream=False,).choices[0].message.contentprint(response)Offline Engine
python
import sglang as sglfrom transformers import AutoProcessor def main(): conversation = [ { 'role': 'user', 'content': [ {'type': 'image'}, {'type': 'text', 'text': 'What appliance can be used to heat food quickly.\nGenerate coordinates for one object bounding box. Constraints: x1,y1,x2,y2 ∈ [0,1000]. Response must be in the format: '}, ], } ] model_path = 'Alibaba-DAMO-Academy/RynnBrain1.1-2B' llm = sgl.Engine(model_path=model_path) processor = AutoProcessor.from_pretrained(model_path) prompt = processor.apply_chat_template( conversation, add_generation_prompt=True, tokenize=False, ) output = llm.generate( prompt=prompt, image_data='cookbooks/assets/object_location/images/000000086408.jpg', sampling_params={"temperature": 0.8, "top_p": 0.95}, ) print(f"Prompt: {prompt}\nGenerated text: {output['text']}") if __name__ == '__main__': main()Check out the cookbooks that showcase RynnBrain's capabilities in cognition, localization, reasoning, and planning.
| Category | Cookbook name | Description |
|---|---|---|
| Spatial Understanding | 1_spatial_understanding.ipynb | Shows the model's ability for spatial understanding in the video scene. |
| Object Understanding | 2_object_understanding.ipynb | Shows how the model understands object categories, attributes, and relations and counting ability. |
| Object Grounding | 3_object_grounding.ipynb | Locates specific objects with bounding boxes in an image or video based on instructions. |
Pretraining & Evaluation
Please refer to RynnScale for details of pretraining and evaluation.
If you find RynnBrain useful for your research and applications, please cite using this BibTeX:
bibtex
@article{damo2026rynnbrain, title={RynnBrain: Open Embodied Foundation Models}, author={Ronghao Dang, Jiayan Guo, Bohan Hou, Sicong Leng, Kehan Li, Xin Li, Jiangpin Liu, Yunxuan Mao, Zhikai Wang, Yuqian Yuan, Minghao Zhu, Xiao Lin, Yang Bai, Qian Jiang, Yaxi Zhao, Minghua Zeng, Junlong Gao, Yuming Jiang, Jun Cen, Siteng Huang, Liuyi Wang, Wenqiao Zhang, Chengju Liu, Jianfei Yang, Shijian Lu, Deli Zhao}, journal={arXiv preprint arXiv:2602.14979v1}, year={2026}, url = {https://arxiv.org/abs/2602.14979v1}} @article{damo2026rynnbrain11, title={RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model}, author={Kehan Li, Bohan Hou, Minghao Zhu, Tianyi Zhang, Zesen Cheng, Zhikai Wang, Sicong Leng, Xin Li, Xiao Lin, Biying Yao, Minghua Zeng, Jiangpin Liu, Ronghao Dang, Jiayan Guo, Siteng Huang, Haoyu Zhao, Heng Ping, Yaxi Zhao, Kexiang Wang, Tong Lu, Shengke Xue, Jiahao Tang, Yulei Wang, Zejing Wang, Jianwei Gao, Shijian Lu, Chengju Liu, Jianfei Yang, Mingxiu Chen, Deli Zhao}, journal={arXiv preprint arXiv:2607.17977}, year={2026}, url = {https://arxiv.org/abs/2607.17977}}| Qwen3.5-122B-A10B |
| Link |
| Link |
| Area Location | 4_area_location.ipynb | Identifies and marks specified regions by points in an image or video. |
| Affordance Location | 5_affordance_location.ipynb | Finds areas or objects with specific affordances in an image or video. |
| Trajectory Location | 6_trajectory_location.ipynb | Infers and annotates trajectories or motion paths in an image or video. |
| 🆕 Contact Point Prediction | 7_contact_point_prediction.ipynb | Predicts an instruction-conditioned contact point and in-plane orientation from an image. |
| 🆕 3D Grounding | 8_3d_grounding.ipynb | Predicts 3D bounding boxes (position, dimensions, orientation) from a single RGB image with camera intrinsics. |