GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Anni0401
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Anni0401
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
| Method | DPO (TRL DPOTrainer), reference model = frozen base model |
| LoRA | r = 8, alpha = 32, dropout = 0.0, target modules q,k,v,o,gate,up,down_proj |
| Hyperparameters | learning rate 1e-5, DPO beta 0.05, 2 epochs, global batch size 32, bf16, gradient checkpointing |
| Sequence length | max 1024 tokens, max prompt 768 tokens |
| Data | 1,114 preference pairs (891 train / 223 validation, split grouped by gold tools) |
Validation preference accuracy (eval_rewards/accuracies) | 0.62 at the last step (0.66 at step 50) |
Queries come from synthetic multi-turn tool-use tasks generated with the repository's multi-agent data generator.
None of the 60 gold tools appears in the Wild-Tool-Bench tool set or in the benchmark tool list.
For each query two rewrites were generated with openai/gpt-oss-120b. Each rewrite was embedded with
Qwen/Qwen3-Embedding-8B and scored by how highly the gold tools are ranked among all candidate tools. The better-scoring
rewrite is chosen, the other is rejected. The dataset is in
data/queries_gold_tools_batch1_dpo_ranked.json.
The adapter expects the rewrite prompt defined in REWRITE_PROMPT_TEMPLATE in
generate_rewrite_pairs_vllm.py
(sampled tool examples + the user query).
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="auto", device_map="auto")tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")model = PeftModel.from_pretrained(base, "Anni0401/qwen3-8b-dpo-query-rewrite-lora")With vLLM: vllm serve Qwen/Qwen3-8B --enable-lora --lora-modules qwen3-8b-dpo-lora=Anni0401/qwen3-8b-dpo-query-rewrite-lora.
The end-to-end effect on tool selection (Wild-Tool-Bench, with gpt-oss-120b and Laguna as executors) is reported in the GitHub
repository, together with all evaluation scripts and logs. The preference accuracy above is not the end-task accuracy.
Qwen3-Embedding-8B).Apache-2.0, inherited from the base model.
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="auto", device_map="auto")tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")model = PeftModel.from_pretrained(base, "Anni0401/qwen3-8b-dpo-query-rewrite-lora")