GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
open-athena
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
open-athena
Model Tree
Qwen/Qwen3-30B-A3B-Base
Input Modalities
Output Modalities
Supported Functionality
Three stages, each one epoch of packed 32,768-token sequences, each starting from the previous stage's weights with a fresh optimizer. All stages use AdamW (adamw_torch_fused; β 0.9/0.95, ε 1e-8), learning rate 5e-6 with cosine decay and no warmup, weight decay 0, gradient clipping at 1.0 and 64 sequences per step. Packed conversations could not attend to each other. The router's auxiliary load-balancing loss (coefficient 0.001) was on, as in Qwen's pretraining. Training used the marin-community/axolotl fork at a208d550 with transformers 5.12.1, FlashAttention 3, fp32 master weights and bf16 compute, on 16 H100s. Conversations longer than 32,768 tokens were dropped. Every stage used the chat template in this repository: Qwen3's template with {% generation %} markers added for training-label masking. The stage-3 settings are in axolotl_config.yaml and axolotl_run_manifest.json.
| Stage | Data | Steps |
|---|---|---|
| 1. Chat | nyu-dice-lab/wildchat50m-rewild-sft-385700 @ 46a5bb5 | 255 |
| 2. Thinking | laion/llama-nemotron-science-reasoning-on-canonical-think-full @ bae881d | 634 |
| 3. Terminal agent | nvidia/Nemotron-Terminal-Corpus @ a1667c4 | 680 (1.43B tokens, of which 0.63B were trained on) |
This export is the bf16 copy of stage 3's final fp32 weights.
The chat template is Qwen3's, with {% generation %} markers added for training. It is in chat_template.jinja and is also embedded in tokenizer_config.json.
text
<|im_start|>system{system}<|im_end|><|im_start|>user{user}<|im_end|><|im_start|>assistant<think>{reasoning}</think> {answer}<|im_end|><|im_end|> (id 151645), which is also the tokenizer's end-of-sequence token, so runtimes stop on it by default. <|endoftext|> (151643) is a second stop.<think> (151667) and </think> (151668). These are ordinary vocabulary tokens, not special tokens, so they stay in the returned text. All evaluation used this default prompt. The template's enable_thinking=False option, which prefills an empty think block, was not tested.content. If a reply's think block closed, the template moves the reasoning out of the content and, as in Qwen's release, drops it from turns before the latest user message. If the think block never closed (common for this model, see below), the template passes the whole <think>… text through unchanged. Send back only the extracted answer instead.tools request field. It lists the function signatures in the system turn and asks for <tool_call>\n{"name": ..., "arguments": {...}}\n</tool_call>. Tool results go in tool messages and are rendered inside a user turn as <tool_response>\n...\n</tool_response>. An assistant message that carries tool_calls must have a string content, such as "", not None. The template checks '</think>' in message.content and raises a TypeError on None, which OpenAI-style clients often send back. <tool_call> and </tool_call> are single tokens (151657, 151658), as vLLM's hermes tool parser expects.generation_config.json sets the sampling used for every evaluation below; vLLM and transformers apply it by default. The token limits are client-side settings, not part of that file.
| Setting | Value |
|---|---|
| temperature | 0.6 |
| top_p | 0.95 |
| repetition_penalty | 1.05 |
| stop | `< |
| max new tokens | 32,768 for single-turn; 16,384 per agent turn |
Stock vLLM and transformers support qwen3_moe. config.json uses the transformers 5 format (rope_parameters), so use transformers>=5 (pip install "transformers>=5" accelerate) and a vLLM release built on it. Evaluation used Marin's vLLM build 0.0.0.dev20260924+marin.01911be34fac on one node of 8 H100 80 GB GPUs, with data parallelism and expert parallelism, a 131,072-token context and up to 32 concurrent sequences. Recent stock vLLM releases should also work but were not tested with this checkpoint.
The native context is 32,768 tokens. Evaluation extended it to 131,072 with YaRN ×4, as Qwen documents for Qwen3, passed at serve time rather than written into config.json. Qwen notes that static YaRN can slightly reduce quality on short texts, so leave it off if you don't need long contexts.
sh
vllm serve open-athena/Qwen3-30B-A3B-Chat-Aligned-Base-2026.10.30 \ --data-parallel-size 8 --enable-expert-parallel \ --max-model-len 131072 --max-num-seqs 32 --max-num-batched-tokens 7168 \ --hf-overrides '{"rope_parameters": {"rope_type": "yarn", "factor": 4.0, "original_max_position_embeddings": 32768, "rope_theta": 1000000.0}, "max_position_embeddings": 131072}' \ --enable-auto-tool-choice --tool-call-parser hermes--max-num-batched-tokens 7168 caps the tokens per scheduler step and was chosen for throughput; it isn't required.
On one 80 GB GPU the 61 GB of weights leave too little room for a 131,072-token KV cache. Serve the native context instead (not tested):
sh
vllm serve open-athena/Qwen3-30B-A3B-Chat-Aligned-Base-2026.10.30 \ --max-model-len 32768 --enable-auto-tool-choice --tool-call-parser hermesAt a 32,768-token context, the prompt and max_tokens must fit together, so lower max_tokens below in that case.
python
from openai import OpenAI def split_reply(text): """Return (reasoning, answer). reasoning is None without a think block; answer is None if the think block never closed (common for this model).""" text = (text or "").lstrip() if not text.startswith("<think>"): return None, text reasoning, closed, answer = text[len("<think>"):].partition("</think>") return reasoning.strip(), (answer.strip() if closed else None) client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")reply = client.chat.completions.create( model="open-athena/Qwen3-30B-A3B-Chat-Aligned-Base-2026.10.30", messages=[{"role": "user", "content": "How many primes are there below 50?"}], max_tokens=32768,)reasoning, answer = split_reply(reply.choices[0].message.content)if answer is None: answer = reasoning # the answer was written inside the unclosed think blockWith transformers, for contexts up to 32,768 tokens:
python
from transformers import AutoModelForCausalLM, AutoTokenizer name = "open-athena/Qwen3-30B-A3B-Chat-Aligned-Base-2026.10.30"tokenizer = AutoTokenizer.from_pretrained(name)model = AutoModelForCausalLM.from_pretrained(name, dtype="bfloat16", device_map="auto")messages = [{"role": "user", "content": "How many primes are there below 50?"}]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=32768 - inputs["input_ids"].shape[1])print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))Evaluation used no vLLM reasoning parser; replies were split as above. vLLM's qwen3 reasoning parser expects a closed think block, so it would misfile this model's unclosed replies as reasoning.
The SFT had to pass a gate before the model was used in the experiment. P_chat is 200 single-turn chat prompts and P_think is 200 olympiad-style reasoning prompts, both at the recommended decode. Replies that hit the 32,768-token limit while still inside their think block count as length-limited and are excluded from the denominators: 0 on P_chat and 33 on P_think, 23 of them repetition loops.
| Check | Threshold | P_chat | P_think |
|---|---|---|---|
| Turn ends on `< | im_end | >` | ≥98% |
| Think block closes, followed by an answer | ≥98% | 35.3% (61 of 173), below threshold | 80.8% (135 of 167), below threshold |
Replies containing </think> / replies opening <think> | — | 65 / 173 | 135 / 200 |
P_think answer accuracy | — | — | 63.5% |
The agentic check, P_agent, runs 32 terminal tasks in the Terminus-2 harness. Each agent turn must be a JSON object with analysis, plan and commands. It was served at 131,072 tokens with YaRN ×4, the hermes tool parser and the recommended decode.
| Check | Threshold | Result |
|---|---|---|
P_agent tasks ending validly (see below) | ≥29 of 32 | 32 of 32, but only 1 completed: 30 hit the time limit, 1 grader failure |
| Agent turns whose JSON parses | ≥95% | 14.9% (499 of 3,356), below threshold |
| Tasks solved | — | 2 of 31 scored |
The model uses the Terminus-2 format but usually breaks the JSON. 1,724 turns contain no complete JSON object (most are cut off mid-object; the median turn is 187 characters). 1,129 contain invalid JSON, mostly bad \ escapes in keystrokes. After the first error, the model often repeats its previous reply verbatim: 1,910 turns do so. Most tasks therefore loop until they time out (30 of 32). A task counts as valid if it completes, hits the harness's 3,000-second agent time limit, or ends with the grader failing; time-outs are expected when small models act as agents, and grader failures are not the model's. Any other harness error counts against the model. So the task count passes, and the broken agent format shows in the parse rate.
The gate also checks the router. For each layer, it compares the share of experts that get under 10% of the mean load against the base model; it allows an increase of at most 2 percentage points.
| Layer | Experts under 10% of mean load (of 128) | Base | Increase (percentage points) |
|---|---|---|---|
| 47 | 16 (12.50%) | 13 (10.16%) | +2.34, over the limit |
| 42 | 9 (7.03%) | 7 (5.47%) | +1.56 |
| 46 | 16 (12.50%) | 14 (10.94%) | +1.56 |
| 44, 45 | 12 (9.38%) | 11 (8.59%) | +0.78 |
One expert is 0.78 percentage points, so layer 47 is one expert over the limit. Measured on the same 2.15M tokens of mixed chat, reasoning and agent text for both models.
All three shortfalls were accepted rather than fixed with extra training.
What this means in use:
</think>. The rest put the whole answer inside the block and end the turn there. Treat everything after <think> as the answer when no </think> follows, as the snippet above does. Reasoning prompts close more often (135 of 200). The final training stage, on terminal-agent data, caused this: after stage 2 the same check gave 89.8% and 96.9%.max_tokens limit.This repository's modifications are released under OpenMDW 1.1; the full text is in LICENSE. The model is a derivative of Qwen/Qwen3-30B-A3B-Base, which Qwen released under the Apache License 2.0. A copy is in LICENSE.Qwen3-Apache-2.0, and the base model's terms continue to apply to the parts derived from it. Modified from the base repository: the weights (by the fine-tune described above), config.json, generation_config.json (sampling defaults) and tokenizer_config.json (end-of-sequence token set to <|im_end|>; chat template embedded). The base repository ships no NOTICE file. The training datasets carry their own licenses; see each dataset's card.
We release our tools to the public with the intent that they are used safely, ethically, and responsibly. They are meant to contribute to the public's understanding of science, and aid in its advancement. By using this model, you agree to comply with all applicable law.
<|im_start|>system{system}<|im_end|><|im_start|>user{user}<|im_end|><|im_start|>assistant<think>{reasoning}</think>{answer}<|im_end|>vllm serve open-athena/Qwen3-30B-A3B-Chat-Aligned-Base-2026.10.30 \ --data-parallel-size 8 --enable-expert-parallel \ --max-model-len 131072 --max-num-seqs 32 --max-num-batched-tokens 7168 \ --hf-overrides '{"rope_parameters": {"rope_type": "yarn", "factor": 4.0, "original_max_position_embeddings": 32768, "rope_theta": 1000000.0}, "max_position_embeddings": 131072}' \ --enable-auto-tool-choice --tool-call-parser hermesvllm serve open-athena/Qwen3-30B-A3B-Chat-Aligned-Base-2026.10.30 \ --max-model-len 32768 --enable-auto-tool-choice --tool-call-parser hermesfrom openai import OpenAI def split_reply(text): """Return (reasoning, answer). reasoning is None without a think block; answer is None if the think block never closed (common for this model).""" text = (text or "").lstrip() if not text.startswith("<think>"): return None, text reasoning, closed, answer = text[len("<think>"):].partition("</think>") return reasoning.strip(), (answer.strip() if closed else None) client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")reply = client.chat.completions.create( model="open-athena/Qwen3-30B-A3B-Chat-Aligned-Base-2026.10.30", messages=[{"role": "user", "content": "How many primes are there below 50?"}], max_tokens=32768,)reasoning, answer = split_reply(reply.choices[0].message.content)if answer is None: answer = reasoning # the answer was written inside the unclosed think blockfrom transformers import AutoModelForCausalLM, AutoTokenizer name = "open-athena/Qwen3-30B-A3B-Chat-Aligned-Base-2026.10.30"tokenizer = AutoTokenizer.from_pretrained(name)model = AutoModelForCausalLM.from_pretrained(name, dtype="bfloat16", device_map="auto")messages = [{"role": "user", "content": "How many primes are there below 50?"}]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=32768 - inputs["input_ids"].shape[1])print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))