GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
switchboardllm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
switchboardllm
Model Tree
Qwen/Qwen2.5-0.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
With Python, using the switchboard-router package (0.7.0 or later):
text
pip install switchboard-routerpython
from switchboard_router import StepRouter router = StepRouter(frontier_threshold=0.2) # Ollama when it's running, else Transformersmodel = router.pick_model(messages, tools) # OpenAI-style chat messages and tool schemasThe router doesn't read raw messages. It reads a compact text made by render_step(messages, tools): the tool names, the task and the latest turns. The render is the same in training and at inference. StepRouter does this for you. If you call the model directly, send that text as the single user message:
python
from switchboard_router import render_stepfrom transformers import AutoModelForCausalLM, AutoTokenizerimport json repo = "switchboardllm/switchboard-agent-router-0.5b"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) text = render_step(messages, tools)ids = tok.apply_chat_template([{"role": "user", "content": text}], add_generation_prompt=True, return_tensors="pt")out = model.generate(ids, max_new_tokens=24, do_sample=False)print(json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)))# {'tier': 'small'}With Ollama: ollama pull hf.co/switchboardllm/switchboard-agent-router-0.5b-GGUF. Then send it the render_step text.
Use greedy decoding (do_sample=False, or temperature 0).
{"tier": "small" | "mid" | "frontier"}. In training:
| Tier | Meaning |
|---|---|
small | Qwen3-1.7B made the reference tool call |
mid | Qwen3-1.7B missed it and Qwen3-8B made it |
frontier | neither did |
Map the tiers to whatever models you run. If the output isn't valid JSON, use frontier: a failed routing call should cost money, never a wrong tool call.
| Build | Size | Runs with | Same pick as this model |
|---|---|---|---|
| This repo (safetensors, BF16) | 988 MB | Transformers, vLLM | reference |
| GGUF F16 | 994 MB | Ollama, llama.cpp | 288 of 300 test steps; the rest were near-ties |
The test set is 2,805 held-out steps from 1,436 tasks the router never saw. Train and test are split by task, so no test task appears in training, not even in another conversation. The steps come from three sources, in their natural mix.
| This router | Random, same tier mix | Source only* | Always frontier | Perfect router | |
|---|---|---|---|---|---|
| Sent too low (needed a bigger tier) | 13.5% | 24.9% | 18.9% | 0% | 0% |
| Cost saved vs. always frontier | 31.4% | 31.4% | 33.7% | 0% | 36.7% |
*Source only always picks the most common tier of the dataset a step came from. It knows something the router isn't told.
Safer settings. With frontier_threshold, StepRouter sends a step to frontier whenever the model gives frontier at least that probability. It never moves a frontier pick down.
frontier_threshold | Sent too low | Cost saved |
|---|---|---|
| none (the model's pick) | 13.5% | 31.4% |
| 0.4 | 13.0% | 31.0% |
| 0.3 | 9.9% | 26.5% |
| 0.2 | 5.8% | 19.2% |
| 0.1 | 2.8% | 11.4% |
Ranking (AUC; 0.5 is chance):
| All | ToolACE | Hermes traces | Nemotron Pivot | |
|---|---|---|---|---|
| Needs frontier vs. not | 0.827 | 0.771 | 0.799 | 0.777 |
| Needs more than small vs. not | 0.820 | 0.753 | 0.820 | 0.790 |
The scores within each source show it reads the step itself, not only which dataset it looks like.
How cost is measured. Output price per 1M tokens of a default model for each tier: 0.50forsmall,10 for mid, $20 for frontier (Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5). It's a proxy for relative cost. The tiers themselves were defined by open Qwen3 models.
| Base | Qwen2.5-0.5B-Instruct (Apache-2.0), full fine-tune in BF16 |
| Parameters | 494M |
| Training data | 23,957 tool-call steps from 12,273 tasks: Team-ACE/ToolACE (Apache-2.0), lambda/hermes-agent-reasoning-traces (Apache-2.0), nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1 (CC-BY-4.0). Mid steps are counted twice in training. |
| Labels | Qwen3-1.7B, then Qwen3-8B, greedy and with thinking off, on each step. The call is compared with the dataset's reference: exact for ToolACE, same tools and argument names for Hermes, exact except long free text for Nemotron. |
| Training | 2 epochs, one A100 80GB, 30 minutes. Loss on the JSON answer only. |
| Input | render_step text, at most 5,000 characters (under 2,048 tokens) |
frontier_threshold where a wrong tool call is expensive.mid is the hardest tier to predict. It is picked often, but it is right on only about a quarter of the steps that need it.Apache-2.0, inherited from Qwen2.5-0.5B-Instruct. Training data includes nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1 by NVIDIA, used under CC-BY-4.0.
from switchboard_router import StepRouter router = StepRouter(frontier_threshold=0.2) # Ollama when it's running, else Transformersmodel = router.pick_model(messages, tools) # OpenAI-style chat messages and tool schemasfrom switchboard_router import render_stepfrom transformers import AutoModelForCausalLM, AutoTokenizerimport json repo = "switchboardllm/switchboard-agent-router-0.5b"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) text = render_step(messages, tools)ids = tok.apply_chat_template([{"role": "user", "content": text}], add_generation_prompt=True, return_tensors="pt")out = model.generate(ids, max_new_tokens=24, do_sample=False)print(json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)))# {'tier': 'small'}