GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
grounded-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
grounded-ai
Model Tree
Cloudflare/clef-flash
Input Modalities
Output Modalities
Supported Functionality
Test set: 1,139 rows from APIs and items never seen in training, sampled to a fixed number of rows per kind of example so rare kinds are measured as precisely as common ones. "Base" is the released Clef-flash on the same rows, zero-shot. Accuracy / expected calibration error, x100.
| Question | Base | agent-decision-9b |
|---|---|---|
| Are the tool calls correct and complete? (yes/no) | 68.0 / 18.5 | 98.1 / 1.0 |
| What is wrong with the calls? (correct / wrong tool / wrong arguments / incomplete) | 56.2 / 8.0 | 96.4 / 1.9 |
| Should the assistant call a tool, ask, or decline? | 60.6 / 22.4 | 88.2 / 3.8 |
| All | 63.3 / 16.6 | 95.3 / 1.4 |
As a cascade's first stage: 91% of decisions come out at 90% confidence or more, and 97.7% of those are right (base: 37%, of which 75% are right).
Are the calls correct? by kind of call (accuracy x100):
| Kind | Base | agent-decision-9b |
|---|---|---|
| Correct calls | 97.3 | 97.3 |
| A parameter the tool does not define | 22.0 | 100 |
| A value the user gave is missing | 45.0 | 97.0 |
| A value differs from what the user asked | 72.2 | 96.5 |
| The wrong tool | 85.3 | 100 |
| One of several requested calls is missing |
Call, ask or decline, by correct answer: call 97.6 → 95.2, decline 58.7 → 87.3, ask 20.7 → 81.1.
Unseen presentation (the same test rows rendered as markdown, a layout never used in training): 93.5% overall (base 75.2%). The weak spot is "ask the user": 44.1% in this layout (base 15.3%).
Needs a GPU with bf16 and about 24 GB (L4 or larger). Qwen3.5's linear attention produces NaNs in float16, so a T4 will not work.
python
import sysfrom huggingface_hub import snapshot_downloadfrom peft import PeftModelfrom safetensors.torch import load_file base = snapshot_download("Cloudflare/clef-flash", revision="8b2e5fd17c09fd49bc2880b805d5323ec7ff4ff3")sys.path.insert(0, base)from joint_schema_model import load_release_model, systemone model, processor = load_release_model(base, device="cuda")ours = snapshot_download("grounded-ai/agent-decision-9b")model.language_model = PeftModel.from_pretrained(model.language_model, ours).merge_and_unload()model.head.load_state_dict(load_file(f"{ours}/joint_head.safetensors")) # calibrated head tools = [{"name": "loginuser", "description": "Log in a user.", "parameters": {"username": {"type": "str"}, "password": {"type": "str"}}}]response = systemone(model, processor, { "model": "agent-decision-9b", "state": {"tools": tools, "user": "Log in a user named 'guest' with the password 'guestpass'.", "assistant_tool_calls": [{"name": "loginuser", "arguments": {"username": "guest789", "password": "guestpass"}}]}, "questions": { "call_ok": {"type": "noul", "instructions": "Are the assistant's tool calls correct and complete for the user's request?"}, "call_kind": {"type": "choice", "instructions": "What, if anything, is wrong with the proposed tool calls?", "criteria": {"correct": "the calls are right and do everything the user asked", "wrong_tool": "a call uses a tool that cannot do what was asked", "wrong_arguments": "the right tool is called with arguments that do not match the request or the tool", "incomplete": "the calls are right but part of the request is not done"}}, },})print(response["answers"])Questions are free-form: any noul (yes/no), choice or score question can be asked about any state, as with Clef. The fine-tune targets the questions above; other questions behave like the base model.
All labels are exact, and nothing is truncated:
52,552 training rows, rendered as JSON or as a transcript; markdown is held out for testing.
LoRA rank 16, alpha 32, no dropout, on every attention, linear-attention and MLP projection of the text decoder except the two small linear-attention gates; the joint head trained fully in fp32. AdamW, learning rate 1e-4 (head 1e-5), constant schedule, effective batch 32 rows (4 length-grouped micro-batches of 8), gradient clipping 1.0, bf16 autocast, gradient checkpointing. The weights with the best validation log loss (step 125 of 150) were kept. One temperature (0.975) fitted on a held-out calibration split and folded into the head. One A100 40 GB, about 25 minutes of training. run_manifest.json holds the exact configuration.
Apache-2.0, like the base model. The training data is CC-BY-4.0: Salesforce xLAM function-calling dataset and NVIDIA When2Call.
| 87.8 |
| 96.3 |