Benchmarks
These are greedy decoding scores. Each row compares base Qwen3.5-4B and Qwenputer on the same
problems. The Python-tool column is base Qwen calling a real Python
interpreter on the host, which is the ceiling for this kind of help.
Table with columns: Benchmark, Base Qwen3.5-4B, Qwenputer, Python tool (host)| Benchmark | Base Qwen3.5-4B | Qwenputer | Python tool (host) |
|---|
| NUPA integer add / sub / mul, 3-20 digits (576) | 48.6 | 100.0 | 100.0 |
| DeepMind arithmetic, 6 sets pooled (600) | 80.8 | 92.3 | 94.2 |
DeepMind div_big (100) | 77 | 97 | 85 |
DeepMind mul_big (100) | 66 | 93 | 91 |
| DeepMind clean big add / mul (200) | 69.5 | 99.0 | 98.5 |
The computer has never printed a wrong result. We replayed about 3.9 million positions of model
output through a reference simulator of the circuit and found zero mismatches. MMLU: 71.9 vs 71.7 for base (99% identical answers, KL 7e-4).
GSM8K and GSM-Hard are ties, because word problems are limited by reasoning rather than
arithmetic. On long DeepMind expression chains, base Qwen is still ahead: Qwen tends to re-verify
exact results until it runs out of its token budget.
The rows are the best mode per benchmark across our development checkpoints. This checkpoint
contains all the features behind them.
Quickstart
git clone https://github.com/divagr18/qwenputer && cd qwenputer && uv sync --extra hfhf download divagr1925/Qwen3.5-4B-Qwenputer-CHIP8 --local-dir qwenputer-4buv run python -m chipformer.q35.demo qwenputer-4b
Every prompt starts with the computer's BIOS. That's a boot record of chip tokens, which loads
the arithmetic library into the computer's memory, followed by Qwen's normal chat template.
After that it's a plain generate() call:
import json, torchfrom transformers import AutoModelForImageTextToText, AutoTokenizerfrom chipformer.q35.demo import TOOLSfrom chipformer.q35.lib import bcdfrom chipformer.ref.emu import Chip8from chipformer.trace import boot_tokens torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False # keep matmuls exact path = "qwenputer-4b"tok = AutoTokenizer.from_pretrained(path)model = AutoModelForImageTextToText.from_pretrained(path, dtype=torch.bfloat16, device_map="cuda")off = json.load(open(f"{path}/chipformer.json"))["chip_token_offset"]bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)] msgs = [{"role": "user", "content": "What is 48213 * 9917?"}]ids = bios + tok.apply_chat_template(msgs, tools=TOOLS, add_generation_prompt=True, enable_thinking=False)["input_ids"]out = model.generate(torch.tensor([ids], device="cuda"), max_new_tokens=4096, do_sample=False, cache_implementation="static")print(tok.decode(out[0, len(ids):], skip_special_tokens=True))
The tools, the system prompts that work best and the optional decoder settings are documented in
docs/running.md.
How it works
Qwen writes "48213 × 9917 ="computer layers read the digits, run the BIOS, emit trace tokensthe model prints " 478128321"Qwen continues as if it had written the number itself
Running. Each generated trace token is one write to the computer's memory. A few tokens make
one instruction, so the trace is the machine's execution log. The MLPs do the arithmetic as exact
lookup tables, and every value stays exact in bf16.
The bridge. The computer layers sit before Qwen's and read Qwen's tokens: digits, operators,
tool-call tags. When a calculation is complete they run it and print the answer as normal text.
Qwen's state is frozen while the computer runs.
What it can do. The computer is about 0.7M nonzero weights covering add, subtract, multiply,
exact division (reduced fractions when it doesn't terminate), signed numbers, mod, gcd, and whole expressions with brackets, powers and decimals.
Division that doesn't terminate prints as a reduced fraction.
Gated DeltaNet as RAM
The computer's RAM is the state of Qwen3.5's own Gated DeltaNet layers. Qwen3.5 mixes full attention with these layers, and each DeltaNet head keeps a state matrix that it updates at every token with the gated delta rule (panel a). In the computer's heads the gate is held open (α = 1, so nothing fades) and the write strength is β = 1. Each key is a one-hot vector picking a single row, so every update replaces exactly that row with the value and leaves the rest untouched. That's a RAM write. A RAM read is the same head's query: a one-hot query returns exactly that row (panel b).
The rest is layout (panel c). The low bits of an address choose the head and the remaining
bits choose the row, and each row stores one byte as bits. The machine's memory is spread over a
few dozen heads, and its registers, program counter and call stack each get heads of their own.
Each key head drives two value heads, which doubles the capacity. The gate is useful too:
α = 0 for one token wipes a whole head (that's how the screen is cleared), and α = ½ makes a head
halve every step, which the circuit uses as a counter.
Writes arrive through the token stream, so the trace is the computer's execution log. An ADDR
token names the row, the VAL token after it carries the value, and STEP ends the instruction.
Nothing here needed a new layer type: the computer's memory is the same Gated DeltaNet
recurrence Qwen3.5 already uses for language.
There is no tool. Qwen can write a calculation inline (48213 × 9917 =), or use the tool-call
syntax it was trained on (<tool_call><function=mul>…). In both cases nothing outside the model
reads it. The <tool_response> that follows is predicted by the model's own layers, token by
token, exactly like every other token. The computer's work shows up in the output as a burst of
special trace tokens, which skip_special_tokens=True hides.
Model details
Table | |
|---|
| Base | Qwen/Qwen3.5-4B, Qwen's own weights unchanged |
| Architecture | stock Qwen3_5ForConditionalGeneration; config-only changes |
| Computer | 12 compiled layers before Qwen's 32; CHIP-8 machine plus bignum BIOS |
| RISC-V version | divagr1925/Qwen3.5-4B-Qwenputer |
| Hidden size | 3200 (Qwen's 2560 plus the computer's dims) |
| Size | 14.65 GB in bf16 (7.3B stored parameters: Qwen3.5-4B's 4.7B, plus zero padding and the computer's layers, of which only 0.7M weights are nonzero) |
| Precision | exact in bf16 |
Citation
@software{agrawal2026qwenputer, author = {Agrawal, Divyansh}, title = {{Qwenputer}: a computer compiled into the weights of a language model}, year = {2026}, version = {0.1.0}, url = {https://github.com/divagr18/qwenputer}, license = {MIT}}
License
Apache-2.0. This is a modified version of
Qwen/Qwen3.5-4B: the config is widened and computer layers
are added, but Qwen's own weights are unchanged.