switchboardllm

switchboard-router-0.5b

README

License: apache-2.0

Run it

With Ollama, one line, no GPU needed:

text

ollama run hf.co/switchboardllm/switchboard-router-0.5b-GGUF "Write a haiku about rain"

json

{"tier":"small","domain":"writing"}

With Transformers:

python

from transformers import AutoModelForCausalLM, AutoTokenizerimport json repo = "switchboardllm/switchboard-router-0.5b"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) msgs = [{"role": "user", "content": "Unscramble the following words: eglar, ngareo, kobo"}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")out = model.generate(ids, max_new_tokens=24, do_sample=False)print(json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)))# {'tier': 'frontier', 'domain': 'general'}

With Python, using the switchboard-router package (no other dependencies; it talks to Ollama, any OpenAI-compatible server, or Transformers):

text

pip install switchboard-router

python

from switchboard_router import Router router = Router()  # uses Ollama when it's runningrouter.pick_model("Write a haiku about rain")# 'anthropic/claude-haiku-5.5'

Use greedy decoding (do_sample=False, or temperature 0).

What it answers

Table with columns: Field, Values
Field	Values
`tier`	`small`, `mid` or `frontier`: the cheapest class of model expected to answer the prompt well
`domain`	`code`, `math`, `writing` or `general`: a hint, from keyword labels

If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.

Builds

Table with columns: Build, Size, Runs with, Same decision as this model
Build	Size	Runs with	Same decision as this model
This repo (safetensors, BF16)	988 MB	Transformers, vLLM	reference
GGUF F16	994 MB	Ollama, llama.cpp	346-347 of 347 test prompts
ONNX 8-bit	660 MB	Transformers.js in the browser	344 of 347 test prompts

Results

Full-precision Switchboard v0.1 on 10,000 held-out prompts it never saw in training, in their natural mix:

Table with columns: Switchboard v0.1, Random router*
	Switchboard v0.1	Random router*
Cost saved vs. always using Claude Opus 5.5	64.5%
Prompts served well (proxy, see below)	94.5%	90.0%
Sent too low (needed frontier, sent cheaper)	5.5%	10.0%
Hard prompts sent to frontier	59.1%	26.2%
Tier accuracy	60.3%
Invalid JSON outputs

*The random router sends the same share of prompts to frontier, picked at random. A perfect router would save 76.2%.

How it's measured. Each prompt in routellm/gpt4_dataset comes with a 2023 Mixtral-8x7B answer scored 1-5 by a GPT-4 judge. A prompt counts as served well if Switchboard sent it to frontier, or sent it lower and that answer scored at least 4. A 2026 small model is stronger than Mixtral-8x7B, so this proxy is cautious. Cost uses Surplus output prices per 1M tokens: Claude Haiku 5.5 ( $0.50) f or s ma l l, C l a u d e S o nn e t 5.5 ($ 10) for mid, Claude Opus 5.5 ($20) for frontier. Every answer is downloadable from the evaluation page.

Versions

Table with columns: v0, v0.1
	v0	v0.1
Cost saved	67.8%	64.5%
Sent too low	7.1%	5.5%
Hard prompts sent to frontier	47.5%	59.1%
Ranking AUC (hard vs. rest)	0.785	0.787

v0.1 trains on 1.5× more frontier examples. It calls the frontier model more often on purpose, giving up 3.3 points of savings for 22% fewer prompts sent too low. Its ability to rank hard prompts above easy ones (AUC) is about the same as v0. v0 is commit 016f5a9 in this repo's history.

The model

Table

Base	Qwen2.5-0.5B-Instruct (Apache-2.0), full fine-tune in BF16
Parameters	494M
Training data	`routellm/gpt4_dataset` (Apache-2.0, LMSYS), 63,749 rows after balancing
Labels	Mixtral score 5 → small, 4 → mid, 1-3 → frontier; `domain` from keywords
Training	2 epochs in 55 minutes; the released weights are the best checkpoint by validation loss (end of epoch 1). One RTX 5070 Ti. Loss on the JSON answer only.
Prompt length	Trained on prompts cut to 512 tokens

Limitations

This is a 0.5B router and it misroutes some prompts: 5.5% of held-out prompts were sent to a cheaper tier than they needed.
Labels come from 2023 models and a GPT-4 judge, not from people. Mostly English prompts.
It reads one prompt at a time, not the whole conversation, and only the first 512 tokens.
A prompt can try to talk it into a cheaper tier. Don't use it as a security boundary.
domain comes from keyword labels and is only a hint.
Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood, Surplus or the Switchboard oracle network.

License

Apache-2.0, inherited from Qwen2.5-0.5B-Instruct and routellm/gpt4_dataset.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

switchboardllm

Model Tree

Base

Qwen/Qwen2.5-0.5B-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Run it

With Ollama, one line, no GPU needed:

text

ollama run hf.co/switchboardllm/switchboard-router-0.5b-GGUF "Write a haiku about rain"

json

{"tier":"small","domain":"writing"}

{"tier":"small","domain":"writing"}

With Transformers:

python

from transformers import AutoModelForCausalLM, AutoTokenizerimport json repo = "switchboardllm/switchboard-router-0.5b"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) msgs = [{"role": "user", "content": "Unscramble the following words: eglar, ngareo, kobo"}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")out = model.generate(ids, max_new_tokens=24, do_sample=False)print(json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)))# {'tier': 'frontier', 'domain': 'general'}

With Python, using the switchboard-router package (no other dependencies; it talks to Ollama, any OpenAI-compatible server, or Transformers):

text

pip install switchboard-router

python

from switchboard_router import Router router = Router()  # uses Ollama when it's runningrouter.pick_model("Write a haiku about rain")# 'anthropic/claude-haiku-5.5'

Use greedy decoding (do_sample=False, or temperature 0).

What it answers

Table with columns: Field, Values
Field	Values
`tier`	`small`, `mid` or `frontier`: the cheapest class of model expected to answer the prompt well
`domain`	`code`, `math`, `writing` or `general`: a hint, from keyword labels

If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.

Builds

Table with columns: Build, Size, Runs with, Same decision as this model
Build	Size	Runs with	Same decision as this model
This repo (safetensors, BF16)	988 MB	Transformers, vLLM	reference
GGUF F16	994 MB	Ollama, llama.cpp	346-347 of 347 test prompts
ONNX 8-bit	660 MB	Transformers.js in the browser	344 of 347 test prompts

Results

Full-precision Switchboard v0.1 on 10,000 held-out prompts it never saw in training, in their natural mix:

Table with columns: Switchboard v0.1, Random router*
	Switchboard v0.1	Random router*
Cost saved vs. always using Claude Opus 5.5	64.5%
Prompts served well (proxy, see below)	94.5%	90.0%
Sent too low (needed frontier, sent cheaper)	5.5%	10.0%
Hard prompts sent to frontier	59.1%	26.2%
Tier accuracy	60.3%
Invalid JSON outputs

*The random router sends the same share of prompts to frontier, picked at random. A perfect router would save 76.2%.

Versions

Table with columns: v0, v0.1
	v0	v0.1
Cost saved	67.8%	64.5%
Sent too low	7.1%	5.5%
Hard prompts sent to frontier	47.5%	59.1%
Ranking AUC (hard vs. rest)	0.785	0.787

The model

Table

Base	Qwen2.5-0.5B-Instruct (Apache-2.0), full fine-tune in BF16
Parameters	494M
Training data	`routellm/gpt4_dataset` (Apache-2.0, LMSYS), 63,749 rows after balancing
Labels	Mixtral score 5 → small, 4 → mid, 1-3 → frontier; `domain` from keywords
Training	2 epochs in 55 minutes; the released weights are the best checkpoint by validation loss (end of epoch 1). One RTX 5070 Ti. Loss on the JSON answer only.
Prompt length	Trained on prompts cut to 512 tokens

Limitations

This is a 0.5B router and it misroutes some prompts: 5.5% of held-out prompts were sent to a cheaper tier than they needed.
Labels come from 2023 models and a GPT-4 judge, not from people. Mostly English prompts.
It reads one prompt at a time, not the whole conversation, and only the first 512 tokens.
A prompt can try to talk it into a cheaper tier. Don't use it as a security boundary.
domain comes from keyword labels and is only a hint.
Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood, Surplus or the Switchboard oracle network.

License

Apache-2.0, inherited from Qwen2.5-0.5B-Instruct and routellm/gpt4_dataset.