GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
GrootTeam
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
GrootTeam
Model Tree
Input Modalities
Output Modalities
Supported Functionality
The chart compares total model parameters with the unweighted average across all 16 benchmarks. Smaller models appear farther left and higher average scores appear higher. Groot-280B-A16B uses 12.5% fewer total parameters than GLM-5.3-Flash and 1.4% fewer than DeepSeek-V4-Flash-0731. It averages 58.85, compared with 58.78 for GLM-5.3-Flash and 55.25 for DeepSeek-V4-Flash-0731, the highest 16-benchmark average among the models shown below 552B.
Accuracy. Groot-280B-A16B-261023 beats both GLM-5.3-Flash and DeepSeek-V4-Flash-0731 on 8 of the 16 benchmarks, with the largest gaps on NL2Repo-Bench (56.71 vs 48.57 / 39.20), SWE-bench Pro (63.47 vs 57.59 / 59.37) and AutomationBench (50.61 vs 46.44 / 40.33).
Cost. On DeepSWE 1.1 it scores 64.2% pass@1 at about 0.13pertask,versus630.24 for GLM-5.3-Flash and 53% at $0.46 for DeepSeek-V4-Flash: higher accuracy at roughly half the cost or less, because it is a smaller model.
The Engram memory (197B parameters, about 197 GB in FP8) is a token lookup table. It adds no activated compute and is included in the release size below.
Available editions:
| Edition | Precision | Size on disk |
|---|---|---|
Groot-280B-A16B-261023 | FP8 + FP4 experts | 365.8 GB |
Groot-280B-A16B-261023-GGUF | UD-Q4_K_XL | 356.2 GB |
Groot-280B-A16B-261023 is a mixture-of-experts model with ~280B total parameters and ~16B activated parameters per token, supporting contexts of up to one million tokens.
The model operates in thinking mode by default and supports a continuously controllable reasoning effort (integer 1 to 100) that trades inference cost for accuracy. All reported results use the maximum effort setting (reasoning_effort=100).
Recommended sampling parameters:
| Parameter | Value |
|---|---|
temperature | 1.0 |
top_p | 0.95 or 1.0 |
context_window | 1M tokens |
max_tokens | ≥ 256K |
For local deployment, we recommend
temperature=1.0,top_p=1.0.
The chat template accepts the following extra kwargs (see chat_template_jinja in config.json):
| Kwarg | Values | Default |
|---|---|---|
thinking_mode | "thinking" | "chat" (enable_thinking is accepted as an alias) | "thinking" |
reasoning_effort | integer in [1, 100], or "low" | "medium" | "high" | "max" | "high" |
drop_thinking | true | false | true |
The string efforts map to budgets low=50, medium=62, high=75, max=100. Assistant reasoning is read from reasoning_content (or thinking as an alias).
shell
vllm serve Groot/Groot-280B-A16B-261023 \ --served-model-name Groot-280B-A16B-261023 \ --tensor-parallel-size 8 \ --max-model-len 1048576 \ --enable-auto-tool-choice \ --tool-call-parser deepseek_v41 \ --reasoning-parser deepseek_v41shell
python -m sglang.launch_server \ --model Groot/Groot-280B-A16B-261023 \ --tp 8 \ --context-length 1048576 \ --reasoning-parser deepseek-v41 \ --tool-call-parser deepseekv41For large-scale deployments, data-parallel attention can be enabled as in the DeepSeek recipes:
shell
python -m sglang.launch_server \ --model Groot/Groot-280B-A16B-261023 \ --tp 8 --dp 8 --enable-dp-attentionThe Chat Completions API can be used with most inference frameworks. Before starting, make sure the SDK is installed and the API key and the API base URL are configured:
shell
pip install -U openai # Set the following accordinglyexport OPENAI_BASE_URL="http://localhost:8000/v1"export OPENAI_API_KEY="EMPTY"python
from openai import OpenAI # Configured by environment variablesclient = OpenAI() messages = [ {"role": "user", "content": "Write a Python function to merge two sorted linked lists."},] chat_response = client.chat.completions.create( model="Groot-280B-A16B-261023", messages=messages, temperature=1.0, top_p=0.95, max_tokens=262144, extra_body={ "chat_template_kwargs": { "enable_thinking": True, # thinking mode; set False for non-thinking "reasoning_effort": 100, # integer budget in [1, 100]; "low"/"medium"/"high"/"max" also accepted "drop_thinking": True, }, },) message = chat_response.choices[0].message# reasoning_content holds the thinking trace; content holds the final answer.print("reasoning:", getattr(message, "reasoning_content", None))print("answer:", message.content)Reasoning effort can also be passed as
low,high,xhigh,max, or a float in [0, 0.99], depending on the serving framework; integer budgets in [1, 100] go throughchat_template_kwargs.
The chat template ships with the checkpoint (chat_template.jinja, also embedded in config.json as chat_template_jinja), so messages can be encoded directly with Transformers, with no separate encoding script:
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "Groot/Groot-280B-A16B-261023"tokenizer = AutoTokenizer.from_pretrained(model_name) messages = [ {"role": "user", "content": "Write a Python function to merge two sorted linked lists."},] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, chat_template_kwargs={ "enable_thinking": True, # thinking mode; set False for non-thinking "reasoning_effort": 100, # integer budget in [1, 100], or "low"/"medium"/"high"/"max" },) model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=8192, temperature=1.0, top_p=0.95)print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))Assistant reasoning is emitted inside the generated text; serving frameworks with a reasoning parser (see --reasoning-parser above) surface it in the separate reasoning_content field instead.
If you use this model in your work, please cite it as:
bibtex
@misc{groot2026, title = {Scaling Down LLMs for Broader Deployment}, author = {The Groot Team}, year = {2026},}vllm serve Groot/Groot-280B-A16B-261023 \ --served-model-name Groot-280B-A16B-261023 \ --tensor-parallel-size 8 \ --max-model-len 1048576 \ --enable-auto-tool-choice \ --tool-call-parser deepseek_v41 \ --reasoning-parser deepseek_v41python -m sglang.launch_server \ --model Groot/Groot-280B-A16B-261023 \ --tp 8 \ --context-length 1048576 \ --reasoning-parser deepseek-v41 \ --tool-call-parser deepseekv41python -m sglang.launch_server \ --model Groot/Groot-280B-A16B-261023 \ --tp 8 --dp 8 --enable-dp-attentionpip install -U openai # Set the following accordinglyexport OPENAI_BASE_URL="http://localhost:8000/v1"export OPENAI_API_KEY="EMPTY"from openai import OpenAI # Configured by environment variablesclient = OpenAI() messages = [ {"role": "user", "content": "Write a Python function to merge two sorted linked lists."},] chat_response = client.chat.completions.create( model="Groot-280B-A16B-261023", messages=messages, temperature=1.0, top_p=0.95, max_tokens=262144, extra_body={ "chat_template_kwargs": { "enable_thinking": True, # thinking mode; set False for non-thinking "reasoning_effort": 100, # integer budget in [1, 100]; "low"/"medium"/"high"/"max" also accepted "drop_thinking": True, }, },) message = chat_response.choices[0].message# reasoning_content holds the thinking trace; content holds the final answer.print("reasoning:", getattr(message, "reasoning_content", None))print("answer:", message.content)import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "Groot/Groot-280B-A16B-261023"tokenizer = AutoTokenizer.from_pretrained(model_name) messages = [ {"role": "user", "content": "Write a Python function to merge two sorted linked lists."},] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, chat_template_kwargs={ "enable_thinking": True, # thinking mode; set False for non-thinking "reasoning_effort": 100, # integer budget in [1, 100], or "low"/"medium"/"high"/"max" },) model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=8192, temperature=1.0, top_p=0.95)print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))@misc{groot2026, title = {Scaling Down LLMs for Broader Deployment}, author = {The Groot Team}, year = {2026},}