GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Loke-60000
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Loke-60000
Model Tree
Input Modalities
Output Modalities
Supported Functionality
bash
vllm serve <path> --served-model-name gl-agent-1-27b \ --max-model-len 262144 \ --enable-auto-tool-choice --tool-call-parser qwen3_xml \ --reasoning-parser qwen3 \ --speculative-config '{"method":"mtp","num_speculative_tokens":1}'Context is 262,144 natively and extensible to 1,000,000 tokens with YaRN. Lower
--max-model-len if your hardware can't hold the full window.
Tool calls are emitted as XML, so --enable-auto-tool-choice and
--tool-call-parser qwen3_xml are both required. Without them tool_calls comes
back null and the raw markup lands in content.
python
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="none") tools = [{"type": "function", "function": { "name": "run_bash", "description": "Run a shell command", "parameters": {"type": "object", "properties": {"cmd": {"type": "string"}}, "required": ["cmd"]}}}] r = client.chat.completions.create( model="gl-agent-1-27b", tools=tools, temperature=1.0, top_p=0.95, messages=[{"role": "user", "content": "The test suite fails. Find out why, start with the CI config."}]) print(r.choices[0].message.tool_calls[0].function)# Function(arguments='{"cmd": "ls -la . && find . -name \\"*.yml\\" ... | head -20"}',# name='run_bash')Feed results back as {"role": "tool", "name": ..., "content": ...} and the model
continues the loop.
The multi-token-prediction head ships inside the weights, so speculative decoding needs no separate draft model. Pass it as shown above.
Measured mean acceptance length is about 1.7 against a ceiling of 2.0 at
num_speculative_tokens: 1, so the drafted token is taken roughly 70% of the
time. The head is one layer deep, which makes 1 its natural setting. Higher
values are accepted but re-run the same layer and acceptance falls off. How much
wall-clock that buys depends on your memory bandwidth, so measure it on your own
hardware rather than trusting a number from someone else's.
On by default. Sampling defaults ship in generation_config.json, and
enable_thinking, preserve_thinking and reasoning_effort are settable per
request through chat_template_kwargs:
python
extra_body={"chat_template_kwargs": {"enable_thinking": False}}Coming soon.
vllm serve <path> --served-model-name gl-agent-1-27b \ --max-model-len 262144 \ --enable-auto-tool-choice --tool-call-parser qwen3_xml \ --reasoning-parser qwen3 \ --speculative-config '{"method":"mtp","num_speculative_tokens":1}'from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="none") tools = [{"type": "function", "function": { "name": "run_bash", "description": "Run a shell command", "parameters": {"type": "object", "properties": {"cmd": {"type": "string"}}, "required": ["cmd"]}}}] r = client.chat.completions.create( model="gl-agent-1-27b", tools=tools, temperature=1.0, top_p=0.95, messages=[{"role": "user", "content": "The test suite fails. Find out why, start with the CI config."}]) print(r.choices[0].message.tool_calls[0].function)# Function(arguments='{"cmd": "ls -la . && find . -name \\"*.yml\\" ... | head -20"}',# name='run_bash')extra_body={"chat_template_kwargs": {"enable_thinking": False}}