GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
SeanWang0027
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
SeanWang0027
Model Tree
Qwen/Qwen3-1.7B
Input Modalities
Output Modalities
Supported Functionality
<|im_end|>; observations, the instruction and the assistant header are context only. Only the teacher's visible reply is used (no hidden reasoning).ScienceWorld test split (200 task variations, AgentGym protocol), 4 independent passes, temperature 0.4, 512 tokens per turn, at most 30 rounds, thinking off, no system turn. Success = final score 100. Mean ± std across the 4 passes.
| model | success rate | Avg@1 (mean final score / 100) |
|---|---|---|
| Qwen3-1.7B (base) | 0.12% ± 0.22 | -0.0331 |
| SFT on gpt-5.4-mini trajectories, 3 epochs | 12.88% ± 1.24 | 0.1557 |
| online ROSE 10+5 with a gpt-5.4-mini teacher, 3 epochs | 17.50% ± 2.29 | 0.2942 |
ROSE minus SFT, paired bootstrap over tasks: +4.62 points success [0.75, 8.75], +0.139 Avg@1 [0.072, 0.203].
The teacher copied the prompt's action template open/close OBJ literally in 29.89% of its turns (the simulator rejects it). This
model reproduces that on 74.88% of its turns and rarely escapes the loop; episodes without it succeed 47.71% of the time. Fixing the
action list in the prompt and re-collecting would likely help more than any training change.
ReAct. The conversation is AgentGym's ScienceWorld instruction as a user turn, a canned assistant acknowledgement, then one user
turn per observation. The model replies Thought:\n...\n\nAction:\n<one command>. Render with Qwen3's chat template with
enable_thinking=False.
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained(REPO); model = AutoModelForCausalLM.from_pretrained(REPO, torch_dtype="bfloat16")ids = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt")Base Qwen3-1.7B scores 0.12%. Every number is the mean ± std of 4 independent passes at temperature 0.4, 512 tokens per turn, at most 30 rounds, thinking off, no system turn; success = final score 100, Avg@1 = mean final score / 100.
| method | epoch | success rate (avg of 4 passes) | Avg@1 | share of turns that are the literal open/close … | model |
|---|---|---|---|---|---|
| SFT | 1 | not evaluated | — | — | qwen3-1.7b-sciworld-sft-gpt54mini-ep1 |
| SFT | 2 | 13.25% ± 2.56 | 0.1574 | 74.20% | qwen3-1.7b-sciworld-sft-gpt54mini-ep2 |
| SFT | 3 | 12.88% ± 1.24 | 0.1557 | 74.88% | qwen3-1.7b-sciworld-sft-gpt54mini-3ep ← this model |
| ROSE 10+5 | 1 | not evaluated | — | — | qwen3-1.7b-sciworld-rose-s10t5-gpt54mini-ep1 |
| ROSE 10+5 | 2 | 7.75% ± 1.92 | 0.1455 | 62.68% | qwen3-1.7b-sciworld-rose-s10t5-gpt54mini-ep2 |
| ROSE 10+5 | 3 | 17.50% ± 2.29 (re-run: 15.38% ± 1.43) | 0.2942 (0.2512) | 35.22% | qwen3-1.7b-sciworld-rose-s10t5-gpt54mini-3ep |
| ROSE 10+5 | 4 | 20.25% ± 2.25 | 0.3433 | 17.98% | qwen3-1.7b-sciworld-rose-s10t5-gpt54mini-ep4 |
| ROSE 10+5 | 5 | 25.75% ± 1.30 | 0.3692 | 2.77% | qwen3-1.7b-sciworld-rose-s10t5-gpt54mini-ep5 |
| ROSE 10+5 | 6 | 20.75% ± 1.82 | 0.2946 | 26.59% | qwen3-1.7b-sciworld-rose-s10t5-gpt54mini-ep6 |
How to read this table:
gpt-5.4-mini, medium) copies the prompt's action template open/close OBJ literally in 29.89% of its own turns; the
simulator rejects it every time. Students inherit the habit, and their test success tracks how strong the habit is in the
particular checkpoint (last column). In online ROSE the habit oscillates during training, so single-checkpoint numbers are
noisy: ROSE epoch 2 was saved at a relapse peak, epochs 3 and 5 near minima, epoch 6 during another relapse.from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained(REPO); model = AutoModelForCausalLM.from_pretrained(REPO, torch_dtype="bfloat16")ids = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt")