GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Shakewin
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Shakewin
Model Tree
Qwen/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
pass@1 is the share of runs that pass, over every task and three samples per task; pass^3 is the share of tasks whose three samples all pass. Test and hard test hold task types that no training set contains. Every choice (data mix, epoch, round) was made on a separate validation split.
| Base | SFT-1 | SFT-2 | EI-3 | |
|---|---|---|---|---|
| Test pass@1 (140 tasks) | 0.419 | 0.774 | 0.738 | 0.745 |
| Test pass^3 | 0.364 | 0.707 | 0.607 | 0.657 |
| Hard test pass@1 (80 tasks) | 0.275 | 0.429 | 0.492 | 0.471 |
| Hard test pass^3 | 0.125 | 0.250 | 0.325 | 0.338 |
| BFCL multi-turn† | 0.360 | 0.285 | 0.325 | 0.395 |
† A local adaptation of BFCL's multi-turn base category, not a leaderboard score. Paired tests, bootstrap intervals and the GRPO pilot: results.
The adapter was trained on prompts rendered by Tadpole's chat template
(agent_from_scratch/prompts/qwen_chat.jinja), with Tadpole's system prompt and tool schemas. It
is measured only inside that harness; with another template or tool format, expect different
behaviour.
With Tadpole and vLLM (one GPU with 24 GB or more), the way every number above was measured. The evaluation guide's server command has two environment settings the GPU image needed:
sh
git clone https://github.com/shakewingo/Tadpole.git && cd Tadpolepip install -e . vllm==0.30.0hf download Shakewin/tadpole-qwen3-4b-sft-2 --local-dir outputs/sft-2vllm serve Qwen/Qwen3-4B-Instruct-2507 --revision cdbee75f17c01a7cc42f958dc650907174af0554 \ --dtype bfloat16 --max-model-len 32768 --enable-lora --max-lora-rank 64 --port 8000 \ --lora-modules SFT-2=outputs/sft-2# in a second shell: the validation split at one seedpython -m agent_from_scratch.evals bench --backend vllm --adapter SFT-2 --split val --seed 0 \ --output outputs/val-sft-2-s0With PEFT:
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen3-4B-Instruct-2507", revision="cdbee75f17c01a7cc42f958dc650907174af0554", dtype="bfloat16")model = PeftModel.from_pretrained(base, "Shakewin/tadpole-qwen3-4b-sft-2")teacher-v2, adding multi-turn demonstrations) repeated 3 times, plus the same 2,000 When2Call and 3,002 Toucan rows: 9,007 rows. Recipe and hashes: mixes/sft-2.json. All rows are in Shakewin/tadpole-sft-data
or can be rebuilt from their public sources, as that card explains.q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj. The loss covers the completion only.Qwen/Qwen3-4B-Instruct-2507 at revision cdbee75f17c01a7cc42f958dc650907174af0554.training_metadata.json has the full record: dataset hash, example IDs, per-epoch adapter
hashes and library versions.One training run per model. A small local benchmark (140 + 80 tasks), not a claim about general agent ability. When2Call scores are in-distribution for every model trained on public data. The adapter inherits the base model's limits; it was trained for tool use inside a workspace, not for general chat.
The adapter is released under Apache-2.0, like its base model, Qwen3-4B-Instruct-2507. It was trained on teacher trajectories generated with DeepSeek-V4.1-Flash, on rows converted from NVIDIA When2Call (CC-BY-4.0) and from Agent-Ark Toucan-1.5M (Apache-2.0). Tadpole's code is MIT.
git clone https://github.com/shakewingo/Tadpole.git && cd Tadpolepip install -e . vllm==0.30.0hf download Shakewin/tadpole-qwen3-4b-sft-2 --local-dir outputs/sft-2vllm serve Qwen/Qwen3-4B-Instruct-2507 --revision cdbee75f17c01a7cc42f958dc650907174af0554 \ --dtype bfloat16 --max-model-len 32768 --enable-lora --max-lora-rank 64 --port 8000 \ --lora-modules SFT-2=outputs/sft-2# in a second shell: the validation split at one seedpython -m agent_from_scratch.evals bench --backend vllm --adapter SFT-2 --split val --seed 0 \ --output outputs/val-sft-2-s0from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen3-4B-Instruct-2507", revision="cdbee75f17c01a7cc42f958dc650907174af0554", dtype="bfloat16")model = PeftModel.from_pretrained(base, "Shakewin/tadpole-qwen3-4b-sft-2")