GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
MMOPD
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
MMOPD
Model Tree
MMOPD/Qwen3-4B-OT3-2ep
Input Modalities
Output Modalities
Supported Functionality
| init | MMOPD/Qwen3-4B-OT3-2ep |
| data | inclusionAI/AReaL-tau2-data tau2_sft_train.jsonl: 32,758 of 33,531 single-turn rows (retail 11,395 / airline 12,320 / telecom 9,043); each row = dialogue context → the next assistant turn (thinking + reply or tool calls) |
| filtering | 251 telecom rows whose task id is in the τ²-bench telecom base or test split were removed (evaluation contamination); 522 airline rows longer than 32,768 tokens were dropped (no truncation of system prompts) |
| rendering | native Qwen3 chat template with the official τ²-bench tool schemas (Tool.openai_schema of each domain) in the system turn; the target is the final assistant turn only: <think>…</think> + text, or <think>…</think> + <tool_call> block; earlier assistant turns in the context carry text + tool calls without thinking (as at inference) |
| recipe | SFT, 3 of 3 epochs (this checkpoint), LR 5e-6 cosine to 5e-7 (5% warmup), 32,768-token flatten packing (8,273 bins per epoch, 64 bins per optimizer step on 64 GPUs), bf16 + ZeRO-2, loss on the assistant target only; ~265M target-context tokens per epoch |
Official harness sierra-research/tau2-bench v1.0.1, telecom base split (114 tasks), 1 trial, user simulator
gpt-4.1-2025-04-14 (temperature 0), agent served by vLLM 0.12 with --tool-call-parser hermes --reasoning-parser qwen3
(the stock Qwen3 parsers), temperature 1.0 / top-p 1.0 / top-k off, 32,768-token context, --max-retries 3 (no retry was
needed in these runs). "Official prompt" = the harness' own LLMAgent system prompt (policy + technical-support manual);
"AReaL prompt" = the shorter system prompt used in the training data, a NON-official protocol. pass^1 in %; "clean" =
pass^1 on the 94 base tasks whose ids do not occur in the SFT data.
| Model | prompt | pass^1 | clean | terminations: user_stop / too_many_errors / max_steps |
|---|---|---|---|---|
| Qwen3-4B-OT3-tau2-1ep | official | 43.9 | 39.4 | 77 / 31 / 6 |
| Qwen3-4B-OT3-tau2-1ep | AReaL | 24.6 | 25.5 | 45 / 67 / 2 |
| Qwen3-4B-OT3-tau2-2ep | official | 57.9 | 55.3 | 98 / 11 / 5 |
| Qwen3-4B-OT3-tau2-2ep | AReaL | 63.2 | 60.6 | 103 / 11 / 0 |
| Qwen3-4B-OT3-tau2-3ep (this) | official | 64.9 | 62.8 | 109 / 5 / 0 |
| Qwen3-4B-OT3-tau2-3ep (this) | AReaL | 63.2 | 58.5 | 103 / 11 / 0 |
An earlier measurement of the 1-epoch and 2-epoch checkpoints used a stricter reasoning parser that rejected agent turns
emitting <think> followed directly by a tool call without </think>; every such turn raised an exception and the harness
re-ran the task from scratch (--max-retries 3), so those numbers include a best-of-up-to-4 effect on format slips and are
NOT comparable with the table above (official prompt / AReaL prompt):
| Model | official (strict parser + retries) | AReaL (strict parser + retries) |
|---|---|---|
| Qwen3-4B-OT3-tau2-1ep | 42.1 | 16.7 |
| Qwen3-4B-OT3-tau2-2ep | 71.9 | 60.5 |
<think> immediately
followed by a <tool_call> without closing the think block (the training data contains 848 turns with empty thinking);
the 2- and 3-epoch checkpoints rarely do. vLLM's stock qwen3 reasoning parser passes such turns through (the tool call is
executed, the stray <think> stays in the message content); a parser that requires a closed think block rejects them.base-split scores are reported both on all 114 tasks and on the 94 tasks whose ids do not occur in the SFT data
(the τ²-bench telecom task ids are deterministic subtask compositions and 20 of the 114 base tasks appear in
AReaL-tau2-data; those rows were excluded from training, but the scenario texts of the full split also underlie the
AReaL RL data, so the whole telecom split should be considered exposed).MMOPD/Qwen3-4B-OT3-{1ep,2ep}, MMOPD/Qwen3-1.7B-OT3-{1ep,2ep} and the
other domain teachers MMOPD/Qwen3-4B-OT3-{medical,law,finance,if}.Serve with vLLM and the Qwen3 tool-call / reasoning parsers, then run the official τ²-bench harness against it:
bash
vllm serve MMOPD/Qwen3-4B-OT3-tau2-3ep --served-model-name student --max-model-len 32768 \ --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser qwen3 --generation-config vllmtau2 run --domain telecom --task-split-name base --agent llm_agent --user user_simulator \ --agent-llm openai/student --agent-llm-args '{"api_base":"http://127.0.0.1:8000/v1","api_key":"EMPTY","temperature":1.0,"top_p":1.0}' \ --user-llm gpt-4.1-2025-04-14 --user-llm-args '{"temperature":0.0}' --num-trials 1 --save-to my_runPlain chat with tools (transformers):
python
from transformers import AutoModelForCausalLM, AutoTokenizerrepo = "MMOPD/Qwen3-4B-OT3-tau2-3ep"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")tools = [{"type": "function", "function": {"name": "get_customer_by_phone", "description": "Look up a customer by phone number.", "parameters": {"type": "object", "properties": {"phone_number": {"type": "string"}}, "required": ["phone_number"]}}}]messages = [{"role": "system", "content": "You are a customer service agent for a telecom company. Follow the policy."}, {"role": "user", "content": "Hi, I can't send MMS messages since this morning. My number is 555-123-2002."}]text = tok.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True, enable_thinking=True)out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=4096, do_sample=True, temperature=1.0, top_p=1.0)print(tok.decode(out[0][tok(text, return_tensors="pt").input_ids.shape[1]:], skip_special_tokens=False))vllm serve MMOPD/Qwen3-4B-OT3-tau2-3ep --served-model-name student --max-model-len 32768 \ --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser qwen3 --generation-config vllmtau2 run --domain telecom --task-split-name base --agent llm_agent --user user_simulator \ --agent-llm openai/student --agent-llm-args '{"api_base":"http://127.0.0.1:8000/v1","api_key":"EMPTY","temperature":1.0,"top_p":1.0}' \ --user-llm gpt-4.1-2025-04-14 --user-llm-args '{"temperature":0.0}' --num-trials 1 --save-to my_runfrom transformers import AutoModelForCausalLM, AutoTokenizerrepo = "MMOPD/Qwen3-4B-OT3-tau2-3ep"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")tools = [{"type": "function", "function": {"name": "get_customer_by_phone", "description": "Look up a customer by phone number.", "parameters": {"type": "object", "properties": {"phone_number": {"type": "string"}}, "required": ["phone_number"]}}}]messages = [{"role": "system", "content": "You are a customer service agent for a telecom company. Follow the policy."}, {"role": "user", "content": "Hi, I can't send MMS messages since this morning. My number is 555-123-2002."}]text = tok.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True, enable_thinking=True)out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=4096, do_sample=True, temperature=1.0, top_p=1.0)print(tok.decode(out[0][tok(text, return_tensors="pt").input_ids.shape[1]:], skip_special_tokens=False))