GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
GrootTeam
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
GrootTeam
Model Tree
Input Modalities
Output Modalities
Supported Functionality
The chart compares total model parameters with the unweighted average across all 16 benchmarks. Smaller models appear farther left and higher average scores appear higher. Groot-238B-A18B uses 16.2% fewer total parameters than DeepSeek-V4-Flash-0731 and 25.6% fewer than GLM-5.3-Flash. It averages 57.89, which is 2.64 points above DeepSeek-V4-Flash-0731 and 0.89 points below GLM-5.3-Flash. At a smaller total size, it stays close to GLM-5.3-Flash's average and leads both on six agentic benchmarks listed below.
Accuracy. Groot-238B-A18B-261023 beats both GLM-5.3-Flash and DeepSeek-V4-Flash-0731 on ALE (47.42 vs 44.45 / 16.53), Toolathlon Verified (71.60 vs 70.06 / 70.37), AutomationBench (47.67 vs 46.44 / 40.33), DeepSWE 1.1 (65.13 vs 63.40 / 40.71), NL2Repo-Bench (50.18 vs 48.57 / 39.20) and BFCL v4 (74.81 vs 74.17 / 73.11).
Cost. On DeepSWE 1.1 it sits in the top-left of the chart below: 65.1% pass@1 at about 0.14pertask,versus630.24 for GLM-5.3-Flash and 53% at $0.46 for DeepSeek-V4-Flash. It is both more accurate and cheaper, because it is a smaller model.
lm_head, routers, some attention projections and the vision tower stay in BF16; a GGUF release is also availableAvailable editions:
| Edition | Precision | Size on disk |
|---|---|---|
Groot-238B-A18B-261023 | FP8 (E4M3) | 250.4 GB |
Groot-238B-A18B-261023-GGUF | UD-Q4_K_XL | 152.2 GB |
Groot-238B-A18B-261023 is a mixture-of-experts model with ~238B total parameters and ~18B activated parameters per token.
The model operates in thinking mode by default: thinking is emitted before the final answer. It supports a 1M-token context window and natively supports tool calling.
reasoning_effort parameter, which accepts three levels: low, high, and max. It defaults to max if not passed (or if set to any other value). To use low or high, pass them explicitly. For benchmark and leaderboard reproduction, keep the default max.clear_thinking defaults to false if not passed. For chat scenarios, explicitly pass clear_thinking=true.Requires vLLM 0.29.0+. The following command serves the model with tensor parallel on 4 GPUs, an FP8 KV cache, and MTP speculative decoding:
shell
vllm serve Groot/Groot-238B-A18B-261023 \ --served-model-name Groot-238B-A18B-261023 \ --tensor-parallel-size 4 \ --kv-cache-dtype fp8 \ --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \ --tool-call-parser glm47 \ --reasoning-parser glm47 \ --enable-auto-tool-choiceshell
sglang serve \ --model-path Groot/Groot-238B-A18B-261023 \ --served-model-name Groot-238B-A18B-261023 \ --tp-size 4 \ --reasoning-parser auto \ --tool-call-parser auto--reasoning-parser auto resolves to the model's thinking parser, so the OpenAI-compatible API places thinking in message.reasoning_content and the final answer in message.content; --tool-call-parser auto returns structured calls in message.tool_calls.
Before starting, make sure the SDK is installed and the API key and the API base URL are configured:
shell
pip install -U openai # Set the following accordinglyexport OPENAI_BASE_URL="http://localhost:8000/v1"export OPENAI_API_KEY="EMPTY"python
from openai import OpenAI # Configured by environment variablesclient = OpenAI() messages = [ {"role": "user", "content": "Write a Python function to merge two sorted linked lists."},] chat_response = client.chat.completions.create( model="Groot-238B-A18B-261023", messages=messages, temperature=1.0, top_p=0.95, extra_body={ "chat_template_kwargs": { "clear_thinking": True, # pass true for chat scenarios "reasoning_effort": "max", # "low" | "high" | "max"; defaults to "max" }, },) message = chat_response.choices[0].message# reasoning_content holds the thinking trace; content holds the final answer.print("reasoning:", getattr(message, "reasoning_content", None))print("answer:", message.content)On follow-up turns, read both
reasoning_contentandcontent. A thinking model can use either field around tool execution.
The model emits well-formed function calls that the server parses into the standard tool_calls field:
python
from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") tools = [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, }] response = client.chat.completions.create( model="Groot-238B-A18B-261023", messages=[{"role": "user", "content": "What is the weather in Paris right now?"}], tools=tools, tool_choice="auto", temperature=1.0, top_p=0.95,) message = response.choices[0].messageprint("reasoning:", getattr(message, "reasoning_content", None))print("answer:", message.content) tool_call = message.tool_calls[0]print(tool_call.function.name, tool_call.function.arguments)# -> get_weather {"city": "Paris"}You can point any OpenAI-compatible SDK (Python, Node.js, etc.) or curl at the same /v1/chat/completions endpoint.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "Groot/Groot-238B-A18B-261023" tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, dtype="auto", device_map="auto",) messages = [{"role": "user", "content": "Give me a short introduction to large language models."}]text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True,)model_inputs = tokenizer([text], return_tensors="pt").to(model.device) generated_ids = model.generate( **model_inputs, max_new_tokens=8192, temperature=1.0, top_p=0.95,)output_ids = generated_ids[0][len(model_inputs.input_ids[0]):]print(tokenizer.decode(output_ids, skip_special_tokens=True))If you use this model in your work, please cite it as:
bibtex
@misc{groot2026, title = {Scaling Down LLMs for Broader Deployment}, author = {The Groot Team}, year = {2026},}vllm serve Groot/Groot-238B-A18B-261023 \ --served-model-name Groot-238B-A18B-261023 \ --tensor-parallel-size 4 \ --kv-cache-dtype fp8 \ --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \ --tool-call-parser glm47 \ --reasoning-parser glm47 \ --enable-auto-tool-choicesglang serve \ --model-path Groot/Groot-238B-A18B-261023 \ --served-model-name Groot-238B-A18B-261023 \ --tp-size 4 \ --reasoning-parser auto \ --tool-call-parser autopip install -U openai # Set the following accordinglyexport OPENAI_BASE_URL="http://localhost:8000/v1"export OPENAI_API_KEY="EMPTY"from openai import OpenAI # Configured by environment variablesclient = OpenAI() messages = [ {"role": "user", "content": "Write a Python function to merge two sorted linked lists."},] chat_response = client.chat.completions.create( model="Groot-238B-A18B-261023", messages=messages, temperature=1.0, top_p=0.95, extra_body={ "chat_template_kwargs": { "clear_thinking": True, # pass true for chat scenarios "reasoning_effort": "max", # "low" | "high" | "max"; defaults to "max" }, },) message = chat_response.choices[0].message# reasoning_content holds the thinking trace; content holds the final answer.print("reasoning:", getattr(message, "reasoning_content", None))print("answer:", message.content)from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") tools = [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, }] response = client.chat.completions.create( model="Groot-238B-A18B-261023", messages=[{"role": "user", "content": "What is the weather in Paris right now?"}], tools=tools, tool_choice="auto", temperature=1.0, top_p=0.95,) message = response.choices[0].messageprint("reasoning:", getattr(message, "reasoning_content", None))print("answer:", message.content) tool_call = message.tool_calls[0]print(tool_call.function.name, tool_call.function.arguments)# -> get_weather {"city": "Paris"}import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "Groot/Groot-238B-A18B-261023" tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, dtype="auto", device_map="auto",) messages = [{"role": "user", "content": "Give me a short introduction to large language models."}]text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True,)model_inputs = tokenizer([text], return_tensors="pt").to(model.device) generated_ids = model.generate( **model_inputs, max_new_tokens=8192, temperature=1.0, top_p=0.95,)output_ids = generated_ids[0][len(model_inputs.input_ids[0]):]print(tokenizer.decode(output_ids, skip_special_tokens=True))@misc{groot2026, title = {Scaling Down LLMs for Broader Deployment}, author = {The Groot Team}, year = {2026},}