BillyWang1

qwen3-8b-tau-sft

README

License: apache-2.0

What "non-thinking" means here

The tokenizer/chat template is a patched Qwen3 template whose assistant branch renders <|im_start|>assistant\n<think>\n\n</think>\n\n + content verbatim, and whose generation prompt ends with that same empty-think prefill. Because </think> is therefore prompt-side, think-relapse is structurally impossible even at temperature 1.0. The patched chat_template.jinja and the embedded copy in tokenizer_config.json ship with this repo, so apply_chat_template reproduces the training format exactly.

Tool calls are emitted as inline Hermes text in the assistant content (the patched template does not render structured tool_calls lists):

markdown
{optional user-visible text}
<tool_call>
{"name": "<fn>", "arguments": {<args object>}}
</tool_call>

At most one call per turn. Tool results are supplied as role: "tool" messages, which the template wraps in <tool_response>…</tool_response> inside a user turn.

Training data

795 reward-1.0 retail trajectories from traj-GLM5 were filtered down to 595 rows over 98 tasks:

airline dropped entirely (it is a held-out zero-shot transfer eval)
12 of 180 synthetic retail tasks dropped by a contamination gate that compares write-action goal signatures against the shipped tau-bench retail task sets. All 12 were retail-train collisions; zero collided with retail-test
135 trajectories dropped for containing a turn with more than one tool call
reasoning traces (reasoning_content) stripped; the tau2 greeting turn dropped

Loss falls on exactly assistant content + <|im_end|>. The assistant prefill header, the inter-turn newline and every observation block are present but masked out, so the training token stream is byte-identical to what an inference server is fed.

Training

Table

base	Qwen/Qwen3-8B
framework	slime 0.3.0 (Megatron), 4×H100
rows / batch	595 / 32
steps	54 (18 per epoch × 3); this repo is epoch 1 = 18 steps
lr	1e-5 cosine, min 1e-6, warmup 0.1
parallelism	TP2, seq-parallel, bf16
loss	token-level SFT on pre-tokenized rows

Final loss 0.118 at epoch 3; this epoch-1 checkpoint was selected because it already passed the boundary-token gate below and further epochs were not needed for priming.

Gates

Boundary-token probe (does SFT actually train the turn-ending specials?):

Table with columns: bucket, n, mean p, median, frac < 0.5
bucket	n	mean p	median	frac < 0.5
`<\|im_end\|>`	64	0.9625	0.9983	1.6%
`<tool_call>` / `</tool_call>`	62	0.9786	0.9998	0.0%
content	7108

Format/geometry gate: converted rows were replayed through the actual multi-turn rollout code with the row's turns scripted as sampled outputs, asserting exact-token prefix alignment of the generation prompt at every turn, loss-token equivalence, tool-parser round-trip, and eos invariants — 595/595 passed. Longest assistant turn: 613 tokens.

Results

Evaluated with greedy decoding on tau-bench retail-test (115 tasks) and airline (50 tasks, never trained on), with a gpt-4.1-mini user simulator.

Table with columns: retail-test, airline
	retail-test	airline
this checkpoint (SFT only)	0.043	0.020
after 32 GRPO steps from this init	0.322	0.060
Qwen3-8B thinking, zero-shot (reference)	0.391	0.240

Sampled (T=1.0, 4 samples/task) after those 32 GRPO steps, where the decoding pathology is absent:

Table with columns: pass@1, pass@2, pass@4
	pass@1	pass@2	pass@4
retail-test	0.315	0.443	0.574
airline (never trained on)	0.140	0.237	0.360

Read the SFT-only number carefully. 0.043 greatly understates the policy: under sampling (T=1.0) the same weights score 0.637 average success on retail-train with pass@16 ≈ 0.97. The gap is a greedy-decoding pathology — 93% of greedy eval episodes exhaust their 30-turn budget in a repetition loop rather than terminating. Subsequent GRPO mostly repaired that pathology (greedy truncation 0.93 → 0.33, repetition 0.48 → 0.15) rather than adding raw capability, which is why greedy retail-test climbs 7.5× while sampled training reward barely moves.

So: this checkpoint is a strong sampled tool-use policy and a weak greedy one. Use sampling, or RL from it.

Intended use and limitations

Research artifact for tau-bench agent RL. Not a general assistant, not tuned for safety or chat. Its system prompt at training time was the synthetic dataset's own <instructions>/<policy> wrapper and its tool schemas were the dataset's tau2-style retail variant, which differ in three tool names from tau-bench's own retail toolset (cancel_order vs cancel_pending_order, find_user_by_contact vs find_user_id_by_email, find_user_by_name vs find_user_id_by_name_zip). In practice the model conditions on whatever schemas are given in context and transferred to the real tau-bench names with essentially no penalty (5 parse failures across 165 eval episodes).

License

Apache-2.0, inherited from both Qwen3-8B and the tau-bench-synthetic dataset.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

BillyWang1

Model Tree

Base

Qwen/Qwen3-8B

Fine-tuned

this model

Input Modalities

Text

Output Modalities