How it was trained
Unlike the other models in the collection, this model trained only the experiment's last SFT stage, terminal-agent data, starting directly from the base. Its pretrained base already closed its think block on every reasoning prompt it finished within 32,768 tokens, so the experiment skipped the chat and thinking stages for it.
Table with columns: Stage, Data, Steps| Stage | Data | Steps |
|---|
| Terminal agent | nvidia/Nemotron-Terminal-Corpus @ a1667c4 | 703 (1.47B tokens, of which 0.65B were trained on) |
- Optimizer: AdamW (
adamw_torch_fused; β 0.9/0.95, ε 1e-8), learning rate 5e-6 with cosine decay and no warmup, weight decay 0, gradient clipping at 1.0.
- Batch: one epoch of packed 32,768-token sequences, 64 sequences per step. Packed conversations could not attend to each other, and conversations longer than 32,768 tokens were dropped.
- Router: the auxiliary load-balancing loss (coefficient 0.001) was on, as in Qwen's pretraining.
- Setup: the
marin-community/axolotl fork at a208d550, transformers 5.12.1, PyTorch SDPA attention, fp32 master weights and bf16 compute, on 16 H100s.
The training settings are in axolotl_config.yaml and axolotl_run_manifest.json. This export is the bf16 copy of the final fp32 weights.
Vision. The checkpoint keeps the base model's vision encoder and image processor (processor_config.json), but training data was text only and image or video input was never evaluated. Evaluation served the model with image and video input disabled.
The chat template is Qwen3.5's, with {% generation %} markers added for training-label masking. It is in chat_template.jinja and is also embedded in tokenizer_config.json.
<|im_start|>system{system}<|im_end|><|im_start|>user{user}<|im_end|><|im_start|>assistant<think>{reasoning}</think> {answer}<|im_end|>
-
Turn end: <|im_end|> (id 248046), which is also the tokenizer's end-of-sequence token, so runtimes stop on it by default. <|endoftext|> (248044) is a second stop.
-
Thinking: the generation prompt prefills <think>\n. A reply therefore starts with the reasoning and contains </think> but no opening <think>. Both tags (248068, 248069) are ordinary vocabulary tokens, so they stay in the returned text.
-
enable_thinking=False prefills an empty think block instead. This model keeps reasoning anyway (see below), so it isn't a way to turn thinking off.
-
Multi-turn: send each earlier reply back as the assistant message's content. The template moves the reasoning out of the content and, as in Qwen's release, drops it from turns before the latest user message.
-
Tools: the template renders the tools request field into the system turn. The model calls tools in Qwen3.5's XML format, which vLLM's qwen3_xml tool parser reads:
<tool_call><function=get_weather><parameter=city>Paris</parameter></function></tool_call>
With transformers' apply_chat_template, give a tool call's arguments as an object (a dict); a JSON string raises TypeError, because the template iterates over it. vLLM's chat API converts OpenAI-style string arguments before rendering. Tool results go in tool messages and are rendered inside a user turn as <tool_response>\n...\n</tool_response>.
Recommended decode
generation_config.json sets the sampling used for every evaluation below, and vLLM and transformers apply it by default. The token limits are client-side settings, not part of that file.
Table with columns: Setting, Value| Setting | Value |
|---|
| temperature | 0.6 |
| top_p | 0.95 |
| repetition_penalty | 1.05 |
| stop | `< |
| max new tokens | 32,768 for single-turn; 16,384 per agent turn |
Serving
qwen3_5_moe requires transformers>=5 (pip install "transformers>=5" accelerate) and a vLLM release that supports Qwen3.5. Evaluation used Marin's vLLM build 0.0.0.dev20260924+marin.01911be34fac on one node of 8 H100 80 GB GPUs, with this command:
vllm serve open-athena/Qwen3.5-35B-A3B-Chat-Aligned-Base-2026.10.31 \ --data-parallel-size 8 --enable-expert-parallel \ --max-model-len 262144 --max-num-seqs 32 --max-num-batched-tokens 7168 \ --gdn-prefill-backend triton \ --limit-mm-per-prompt '{"image": 0, "video": 0}' \ --enable-auto-tool-choice --tool-call-parser qwen3_xml
--gdn-prefill-backend triton selects the prefill kernel for the Gated DeltaNet layers in the Marin build. If your vLLM doesn't recognize the flag, leave it out.
--limit-mm-per-prompt disables image and video input.
--max-num-batched-tokens 7168 was chosen for throughput, not correctness.
Recent stock vLLM releases should also work but were not tested with this checkpoint. On a single 80 GB GPU, the 70 GB of weights leave little room for a KV cache. Drop --data-parallel-size and --enable-expert-parallel and shrink --max-model-len, for example to 65,536 (not tested). vLLM rejects a request whose prompt plus max_tokens exceeds --max-model-len, so lower max_tokens in the snippet below to fit.
from openai import OpenAI def split_reply(text): """Return (reasoning, answer). The template prefills <think>, so the reply starts inside the think block; answer is None if it never closed.""" text = text or "" reasoning, closed, answer = text.partition("</think>") return reasoning.removeprefix("<think>").strip(), (answer.strip() if closed else None) client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")reply = client.chat.completions.create( model="open-athena/Qwen3.5-35B-A3B-Chat-Aligned-Base-2026.10.31", messages=[{"role": "user", "content": "How many primes are there below 50?"}], max_tokens=32768,)reasoning, answer = split_reply(reply.choices[0].message.content)
With transformers, text only. Load it with AutoModelForImageTextToText, the class this checkpoint was saved as, not AutoModelForCausalLM. Allow more than 80 GB of GPU memory (two GPUs). Install flash-linear-attention and causal-conv1d for fast Gated DeltaNet kernels; without them generation falls back to a slow path. Training used transformers 5.12.1:
from transformers import AutoModelForImageTextToText, AutoTokenizer name = "open-athena/Qwen3.5-35B-A3B-Chat-Aligned-Base-2026.10.31"tokenizer = AutoTokenizer.from_pretrained(name)model = AutoModelForImageTextToText.from_pretrained(name, dtype="bfloat16", device_map="auto")messages = [{"role": "user", "content": "How many primes are there below 50?"}]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=32768)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Evaluation used no vLLM reasoning parser; replies were split as above.
Evaluation and known behavior
All results below use the recommended decode.
P_chat: 200 single-turn chat prompts.
P_think: 200 olympiad-style reasoning prompts.
- Replies that hit the 32,768-token limit while still inside their think block count as length-limited and are excluded from the denominators.
Protocol, with thinking on (the template's default):
Table with columns: Check, Threshold, P_chat, P_think| Check | Threshold | P_chat | P_think |
|---|
| Turn ends on `< | im_end | >` | ≥98% |
| Think block closes, followed by an answer | ≥98% | 100% (200 of 200) | 100% (194 of 194) |
| Length-limited (of which repetition loops) | — | 0 | 6 (1) |
P_think answer accuracy | — | — | 84.0% |
At repetition penalty 1.0, the same checks also pass; P_think accuracy is 82.0%.
Agentic check, P_agent. It runs 32 terminal tasks in the Terminus-2 harness, served at 262,144 tokens with the qwen3_xml tool parser.
Table with columns: Check, Threshold, Result| Check | Threshold | Result |
|---|
| Tasks ending validly (see below) | ≥29 of 32 | 32 of 32: 24 completed, 7 hit the time limit, 1 grader failure |
| Tool calls that parse | ≥95% | 96.9% (936 of 966 turns) |
| Tasks solved | — | 18 of 31 scored (58%) |
A task counts as valid if it completes, hits the harness's 3,000-second agent time limit, or ends with the grader failing; time-outs are expected when small models act as agents, and grader failures are not the model's. Any other harness error counts against the model.
Routing. The gate also checks the router. For each layer, it compares the share of experts that get under 10% of the mean load against the base model, and allows an increase of at most 2 percentage points. This model passes. Its worst layers, 38 and 39, are at +1.56 points; in layer 38, that's 4 experts of 256, against none in the base. The check used the same 2.2M tokens of mixed chat, reasoning and agent text for both models.
What this means in use:
- Agent runs can be slow. Seven of the eight incomplete
P_agent tasks hit the 3,000-second agent time limit. Six of them were long, reasoning-heavy runs with few format errors. One was a repetition loop, in which 105 of 155 turns repeated the previous reply. The eighth was a timeout of the task's grader, not the model. Allow generous time limits, cap reasoning per turn, and watch for repeated turns.
enable_thinking=False doesn't stop reasoning. With thinking switched off, the model still reasons and closes a </think> on 197 of the 200 reasoning prompts at repetition penalty 1.05, and 187 of 200 at 1.0. At 1.0, 13 replies ran to the token limit without ending the turn (turn end 93.5%); at 1.05, turn end is 98.0%. The pretrained base behaves the same way. Plan on thinking always being on.
- Only the checks above were run. There is no broad capability evaluation.
License
The weights are governed by two licenses. This repository's modifications are released under OpenMDW 1.1; the full text is in LICENSE. The model is a derivative of Qwen/Qwen3.5-35B-A3B-Base, which Qwen released under the Apache License 2.0. A copy is in LICENSE.Qwen3.5-Apache-2.0, and the base model's terms continue to apply to the parts derived from it.
Changed from the base repository:
- the weights, by the fine-tune described above;
config.json: dtype, end-of-sequence and padding token ids, and the transformers 5 format;
generation_config.json: sampling defaults;
tokenizer_config.json: end-of-sequence token set to <|im_end|>, transformers 5 special-token fields, and the chat template embedded;
chat_template.jinja: {% generation %} markers added.
- The base's multi-token-prediction (MTP) weights are not included.
config.json still lists one MTP layer, but no mtp.* tensors are in the checkpoint, so MTP speculative decoding is unavailable. The fine-tune trained only the main model.
The base repository ships no NOTICE file. The training dataset carries its own license; see its card.
We release our tools to the public with the intent that they are used safely, ethically, and responsibly. They are meant to contribute to the public's understanding of science, and aid in its advancement. By using this model, you agree to comply with all applicable law.