JaspinXu

KnowWhenToHandOff-Qwen3-4B-R2

README

License: apache-2.0

Model family

Table with columns: Model, Training, Hand-off F1, Over-escalation, Use it for
Model	Training	Hand-off F1	Over-escalation	Use it for
SFT (B2)	LoRA SFT on 790 filtered teacher dialogues	0.768	0.016	starting point; precise but misses hand-offs
R1	GRPO from B2, task reward only	0.802	0.462	studying the outcome-reward loophole
R2	GRPO from B2, task reward + violation and hand-off penalties	0.812	0.103	the best-calibrated hand-off model

Task success on the official test tasks is the same for all three within noise (0.33–0.35).

Quick start

A standard Qwen3 chat model with tool calling. It was trained with the τ²-bench domain policy as the system prompt and the domain's tools, including transfer_to_human_agents; for faithful behaviour run it inside the evaluation harness of the code repository.

python

from transformers import AutoModelForCausalLM, AutoTokenizer repo = "JaspinXu/KnowWhenToHandOff-Qwen3-4B-R2"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(    repo, torch_dtype="auto", device_map="auto") transfer = {    "type": "function",    "function": {        "name": "transfer_to_human_agents",        "description": "Transfer the user to a human agent.",        "parameters": {            "type": "object",            "properties": {"summary": {"type": "string"}},            "required": ["summary"],        },    },}messages = [    {"role": "system", "content": "<airline or retail policy>"},    {"role": "user", "content": "I need the unaccompanied-minor service."},]inputs = tokenizer.apply_chat_template(    messages, tools=[transfer], add_generation_prompt=True,    return_tensors="pt",).to(model.device)output = model.generate(inputs, max_new_tokens=512)print(tokenizer.decode(output[0][inputs.shape[-1]:], skip_special_tokens=True))

Serve it with vLLM (OpenAI-compatible API, tool calls parsed):

bash

vllm serve JaspinXu/KnowWhenToHandOff-Qwen3-4B-R2 \  --enable-auto-tool-choice --tool-call-parser hermes

Evaluation

Test split, protocol frozen in ADR-005; 95 % bootstrap intervals over tasks.

Table with columns: Metric, This model, B2 (SFT start), Source
Metric	This model	B2 (SFT start)	Source
Success (pass^1), test_main	0.342 [0.245, 0.449]	0.327 [0.224, 0.434]	main table
Paired Δ success vs B2	+0.015 [−0.056, 0.082]	—	paired differences
pass^4	0.122 [0.041, 0.224]	0.102 [0.041, 0.204]

Known behaviour. R2 removes R1's over-escalation but sometimes makes the opposite error on out-of-scope requests: it refuses and tells the user to contact the company directly without calling the transfer tool, or substitutes an unrelated write (under-escalation 0.25). Once it justified a hand-off with a policy that does not exist.

Training details

Base model: Qwen/Qwen3-4B-Instruct-2507 (revision cdbee75f17c01a7cc42f958dc650907174af0554), Apache-2.0.
Training: GRPO from the SFT checkpoint B2 with the composite reward (configs/reward/r2.yaml: task reward − 0.25 × weighted violations − 0.5 × unneeded transfer − 1.0 × missed transfer − 0.1 × format errors); LoRA rank 64, alpha 128, lr 1e-5; 8 tasks × 8 rollouts per step, 60 steps, no KL term.
Training run R2-s1-20261006 (seed 1); exported step-60 checkpoint sha256:be6198a0c51773f52ba4f20c9605a5931f23b0206e92a9fe0521a93761348c96 (merged bf16, Hugging Face format).
Evaluation code at git 462b5d17b4e9add637a2ee7802974c720b625a5b; τ²-bench fc0055dc4e0a316c3f83133267fbd6faaa770992; all sources in reports/main/sources.json.

Intended use

Studying when a tool-using customer-service agent should hand off to a human, in the τ²-bench airline and retail domains. Out of scope: any production or customer-facing use, other domains, and decisions about real people.

Training data

Simulated conversations on τ²-bench v1.0.1 train-split tasks (a dev split was carved out for selection) and hand-off variants derived from them (explicit requests for a human, out-of-scope requests, hard negatives); the user simulator is Qwen3-30B-A3B-Instruct-2507. No real user data. Test tasks were never used for training or selection.

Limitations

One user simulator (Qwen3-30B-A3B-Instruct-2507, which also generated the SFT data) and one training seed; results may not transfer to real users or other simulators, and the intervals cover task sampling, not training variance.
Small test split (49 official tasks; retail uses the 74 tasks whose reward needs no LLM judge, so retail numbers are not comparable with leaderboards).
Communication styles describe how users write, not who they are; no claim is made about any demographic group.
Policy compliance is checked by five deterministic rules that cover only part of each domain policy; citation rate is a keyword rule (it counted one invented policy as a citation).
Observed reward-hacking behaviours: docs/rl-audit.md.

Licences

The weights are a LoRA fine-tune of Qwen/Qwen3-4B-Instruct-2507 and inherit its Apache-2.0 licence. τ²-bench code and data (pinned commit fc0055dc4e0a316c3f83133267fbd6faaa770992) are used under the licence in its LICENSE file; released dialogue samples are τ²-bench-derived synthetic conversations with no real user data.

Citation

Model DOI: 10.57967/hf/10823. Cite the model and the project:

bibtex

@misc{knowwhentohandoff2026r2,  title     = {KnowWhenToHandOff-Qwen3-4B-R2},  author    = {{KnowWhenToHandOff contributors}},  year      = {2026},  publisher = {Hugging Face},  doi       = {10.57967/hf/10823},  url       = {https://huggingface.co/JaspinXu/KnowWhenToHandOff-Qwen3-4B-R2}} @misc{knowwhentohandoff2026,  title  = {Know When to Hand Off: Multi-turn Reinforcement Learning for            Trustworthy Customer-Service Agents},  author = {{KnowWhenToHandOff contributors}},  year   = {2026},  howpublished = {\url{https://github.com/JaspinXu/KnowWhenToHandOff}}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

JaspinXu

Model Tree

Base

JaspinXu/KnowWhenToHandOff-Qwen3-4B-SFT

Fine-tuned

this model

Input Modalities

Text

Output Modalities