GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
dari-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
dari-ai
Model Tree
Qwen/Qwen3.6-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
The intended interface is an authenticated, OpenAI-compatible /v1/chat/completions endpoint serving the adapter. This complete example randomizes two candidates into anonymous labels, sends only those labels to the policy, validates the response, and recovers the selected candidate.
python
import jsonimport osimport randomfrom openai import OpenAISYSTEM_PROMPT = ("You are Dari's model router for a coding agent. ""Choose exactly one anonymous candidate action for the next agent turn. ""Use the visible conversation, benchmark evidence, previous action, and projected costs. ""Prefer a cheaper action only when it is sufficiently likely to complete the task successfully. ""Return only JSON matching the provided schema.")candidate_records = [{"candidate": {"model": "provider/strong-model", "reasoning_effort": "high"},"eval_score": 82.0,"cost": {"warm_tokens": 0,"est_prompt_tokens": 2400,"est_input_cost_usd": 0.0024,"output_cost_per_mtok": 10.0,"pricing_known": True,"fixed_turn_cost_estimate": {"projected_turns": 10,"output_tokens_per_turn": 400,"assumed_reasoning_effort": "high","total_cost_usd": 0.064,},},},{"candidate": {"model": "provider/fast-model", "reasoning_effort": "medium"},"eval_score": 76.0,"cost": {"warm_tokens": 0,"est_prompt_tokens": 2400,"est_input_cost_usd": 0.0008,"output_cost_per_mtok": 2.0,"pricing_known": True,"fixed_turn_cost_estimate": {"projected_turns": 10,"output_tokens_per_turn": 250,"assumed_reasoning_effort": "medium","total_cost_usd": 0.013,},},},]random.SystemRandom().shuffle(candidate_records)action_to_record = dict(zip(["A", "B"], candidate_records, strict=True))policy_input = {"candidate_actions": [{"action": action} for action in action_to_record],"imported_evals": [{"id": "coding-eval","name": "Coding Eval","description": "Verified coding-task success rate.","min_score": 0,"max_score": 100,"scores": [{"action": action, "score": record["eval_score"]}for action, record in action_to_record.items()],}],"previous_action": None,"cost_estimates": [{"action": action, **record["cost"]}for action, record in action_to_record.items()],"messages": [{"role": "user", "content": "Fix the race condition and add a regression test."}],}client = OpenAI(base_url=os.environ["DARI_ROUTER_BASE_URL"],api_key=os.environ["DARI_ROUTER_API_KEY"],)response = client.chat.completions.create(model=os.environ.get("DARI_ROUTER_MODEL", "router-grpo-fullctx150-nf4"),messages=[{"role": "system", "content": SYSTEM_PROMPT},{"role": "user", "content": json.dumps(policy_input, separators=(",", ":"))},],temperature=0,response_format={"type": "json_schema","json_schema": {"name": "anonymous_routing_action","strict": True,"schema": {"type": "object","additionalProperties": False,"properties": {"action": {"type": "string","enum": list(action_to_record),}},"required": ["action"],},},},)content = response.choices[0].message.contentselection = json.loads(content)if set(selection) != {"action"} or selection["action"] not in action_to_record:raise ValueError("Router policy returned an invalid action")selected_candidate = action_to_record[selection["action"]]["candidate"]print(selected_candidate)
The candidate names, scores, and prices above are illustrative placeholders. Replace them with one internally consistent record per candidate, then derive every action-keyed field from the same shuffled mapping as the example does.
Set DARI_ROUTER_BASE_URL to the authorized service URL ending in /v1, and set DARI_ROUTER_API_KEY to its bearer token. The URL and token are intentionally not published in this model card; request them from the Dari operator responsible for the deployment. Access to the Hugging Face repository and access to a serving endpoint are separate permissions. The default model ID belongs to the known NF4 deployment; other deployments should set DARI_ROUTER_MODEL to the ID returned by their /v1/models endpoint.
The serving contract accepts non-streaming requests with temperature=0, n=1, and the strict JSON Schema shape shown above; only the action enum changes per request. The assistant message content is compact JSON such as {"action":"C"}; it is not the entire chat-completions response. Unsupported fields, malformed schemas, invalid labels, non-JSON completions, and non-success HTTP responses must fail explicitly.
Candidate labels use a unique, contiguous spreadsheet-style sequence: A through Z, then AA, AB, and so on. Candidates are shuffled into that sequence for every request, and the real mapping remains outside the policy prompt. The current serving contract accepts between 2 and 700 syntactically valid labels. Training used 12 choices per request, and protocol validation covered requests with exactly 7 choices. Prefer a validated 7- or 12-choice shape; the larger ceiling only prevents a transport failure and is not evidence of routing quality at 700 choices.
previous_action is the prior validated policy choice for the same routing session, or null when none exists. warm_tokens estimates the downstream candidate's reusable cached-prefix tokens; est_prompt_tokens and est_input_cost_usd estimate its next downstream request. fixed_turn_cost_estimate.total_cost_usd is the preferred comparison: it projects repeated input and output cost across the stated number of future turns. Inputs with unknown prices should set pricing_known to false and omit unsupported cost projections. Evaluation scores must use the declared minimum/maximum scale with higher values representing better performance.
The example below loads the v40 adapter (the final checkpoint) with its exact base-model revision. The known deployment uses an H100 80 GB GPU and 4-bit NF4 quantization; other hardware has not been validated.
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigBASE_MODEL = "Qwen/Qwen3.6-35B-A3B"BASE_REVISION = "995ad96eacd98c81ed38be0c5b274b04031597b0"ADAPTER_MODEL = "dari-ai/router-slm"ADAPTER_REVISION = "dd677af3717edcb52991a100fa56aaa4ed6ffd35" # v40 (final)# v33 (benchmarked): "eda01bf907bdf10556d60dbaaf56062ba276319d"quantization = BitsAndBytesConfig(load_in_4bit=True,bnb_4bit_quant_type="nf4",bnb_4bit_compute_dtype=torch.bfloat16,)tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, revision=BASE_REVISION)base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL,revision=BASE_REVISION,device_map="auto",quantization_config=quantization,)model = PeftModel.from_pretrained(base_model,ADAPTER_MODEL,revision=ADAPTER_REVISION,)model.eval()
The deployment that produced this adapter used Python 3.10, CUDA 13.0.2, PyTorch 2.12.1, Transformers 5.13.0, PEFT 0.18.1, bitsandbytes 0.49.2, and Accelerate >=1.10,<2. Loading the weights does not provide constrained decoding by itself; unconstrained generate() is not the supported routing interface. This repository publishes weights and the wire contract, not a standalone self-hosting server.
Each policy checkpoint has an immutable Hugging Face tag. v40 is the recommended default; main currently matches it, but pin the tag, not main. Earlier tags are retained so existing deployments can pin the exact artifact they validated against.
| Tag | Hugging Face revision |
|---|---|
v3 | 7679ad51ee216116be883cc357c15aa5321c9311 |
v5 | a9248cc66aa572b90f84619254bc349be9944399 |
v33 | eda01bf907bdf10556d60dbaaf56062ba276319d |
v40 | dd677af3717edcb52991a100fa56aaa4ed6ffd35 |
Tags are ordered by recency. End-to-end router results depend on the candidate set, prompt state, provider behavior, and serving configuration; this card makes no cross-version performance claims.
To verify adapter bytes:
bash
hf download dari-ai/router-slm adapter_model.safetensors \--revision v40 --local-dir router-grpo-v40sha256sum router-grpo-v40/adapter_model.safetensors# v40: ed915135e2e420c056e78d0e397090ca22ec19df8ce58fa604aca6d50e9b8dc7# v33: 26f72bf95f3ea934e1d6d37ed0117e317619c963f4353f3294f4221ae6ecddc2
The policy is a LoRA adapter (rank 32, alpha 64) on Qwen/Qwen3.6-35B-A3B at revision 995ad96eacd98c81ed38be0c5b274b04031597b0. Training details are internal. The repository contains inference adapters only: no optimizer state, no trajectory data, no resumable training checkpoints.
The intended use is model-and-reasoning-level selection for coding agents inside the Dari router. The policy was trained on grouped SWE-bench coding trajectories. Other task distributions, prompt formats, unconstrained decoding, or substantially different candidate sets can produce unreliable decisions.
The policy only chooses among candidates supplied by the caller. The surrounding router remains responsible for eligibility, provider authorization, availability, fallback behavior, observability, user-data handling, and rejection of invalid policy output.
No separate adapter license is currently published in this private repository. Confirm permitted use with the repository owner and comply with the terms attached to the pinned Qwen base model. Do not include provider credentials, API keys, or other secrets in policy prompts.