JonathanColetti

Qwen3.5-0.8B-Astra-WeightEdit

README

License: apache-2.0

What changed

The loop direction was subtracted from 64 output columns of mlp.down_proj in layers 15 and 19 (about 130k weights, 0.016% of the model). Everything else is byte-identical to the base model.

How to run

[!IMPORTANT] The edit only helps with thinking on. The chat template turns thinking off unless you pass enable_thinking=True. Recommended sampling (the paper's): temperature 0.6, top_p 0.95, top_k 20, presence penalty 1.5.

Table with columns: You want to run it in…, Use, Files
You want to run it in…	Use	Files
Python, GPU servers	🤗 Transformers / vLLM / SGLang	`model.safetensors-*` (bf16)
The browser (WebGPU) or Node.js	🤗 Transformers.js	`onnx/*`
Any ONNX Runtime app (C#, C++, Java, mobile…)	ONNX Runtime	`onnx/*`
A laptop / CPU / Mac, local apps	llama.cpp, LM Studio, Ollama, Jan…	`Qwen3.5-0.8B-Astra-WeightEdit-BF16.gguf`

python

from transformers import AutoModelForImageTextToText, AutoTokenizerrepo = "JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForImageTextToText.from_pretrained(repo, dtype="bfloat16", device_map="auto")msgs = [{"role": "user", "content": "What is 17 * 23? Please reason step by step, and put your final answer within \\boxed{}."}]inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True,                                 return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=4096, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

bash

vllm serve JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit# then send chat requests with: "chat_template_kwargs": {"enable_thinking": true}

bash

npm i @huggingface/transformers@next

import { AutoProcessor, Qwen3_5ForConditionalGeneration, TextStreamer } from "@huggingface/transformers"; const repo = "JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit";const processor = await AutoProcessor.from_pretrained(repo);const model = await Qwen3_5ForConditionalGeneration.from_pretrained(repo, {  dtype: { embed_tokens: "fp16", vision_encoder: "fp16", decoder_model_merged: "fp16" },  device: "webgpu", // or "wasm" in the browser, "cpu" in Node.js (use "quantized" there)});const messages = [{ role: "user", content: "What is 17 * 23? Please reason step by step, and put your final answer within \\boxed{}." }];const text = processor.apply_chat_template(messages, { add_generation_prompt: true, enable_thinking: true });const inputs = await processor(text);await model.generate({  ...inputs, max_new_tokens: 4096, do_sample: true, temperature: 0.6, top_p: 0.95, top_k: 20,  streamer: new TextStreamer(processor.tokenizer, { skip_prompt: true, skip_special_tokens: true }),});

Which dtype to pick (each graph comes in all five; size is the language-model decoder):

Table with columns: dtype, decoder size, Best for, Agrees with bf16*
dtype	decoder size	Best for	Agrees with bf16*
`fp32`	3.0 GB	reference	100%
`fp16`	1.5 GB	WebGPU	100%
`quantized` (int8)	0.87 GB	CPU / WASM	100%
`q4f16`	0.44 GB	smallest download, WebGPU	~69%
`q4`	0.48 GB	smallest download, CPU / WASM	~69%

*Next-token argmax agreement with the PyTorch model over a 35-token prompt (spot check). The 4-bit builds lose about as much against the bf16 base model as they do here, so treat them as noticeably lossy at this size.

bash

# llama.cppllama-cli -hf JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit:BF16 \  --jinja --chat-template-kwargs '{"enable_thinking":true}' \  --temp 0.6 --top-p 0.95 --top-k 20 --presence-penalty 1.5 -c 32768 # Ollama (a version with Qwen3.5 support)ollama run hf.co/JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit:BF16

LM Studio: search for JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit. Only a BF16 GGUF is provided (1.56 GB). This is text only: no mmproj vision file.

How the ONNX and GGUF files were made

Neither was re-exported. I took existing conversions of the same base revision and rewrote only the two edited tensors, so every other byte matches those well-tested builds:

ONNX: onnx-community/Qwen3.5-0.8B-ONNX-OPT @ 9a4c21d8. The down_proj tensors of layers 15 and 19 are overwritten in place in each decoder_model_merged* data file. Before writing, the script regenerates the base tensors from the bf16 weights and requires a byte-for-byte match with the shipped files in all five dtypes, including the int8/int4 block quantization. So the edited bytes are exactly what that export pipeline would have produced. embed_tokens* and vision_encoder* are unchanged copies.
GGUF: ggml-org/Qwen3.5-0.8B-GGUF Qwen3.5-0.8B-BF16.gguf @ 807dfd9a, converted from Qwen revision 2fc06364. blk.{15,19}.ffn_down.weight are swapped after checking that the base bytes match the base sha256 in astra_edit/manifest.json. The edited bytes match the edited sha256. Only general.name, general.repo_url and the base-model metadata were changed.
Checks: the ONNX fp32/fp16/int8 logits match PyTorch (mean KL ≤ 6e-4, same argmax). Transformers.js fp32 greedy output is token-for-token identical to PyTorch over 120 tokens.
Scripts: conversion/.

The root config.json gained a transformers.js_config block, and a minimal generation_config.json (eos_token_id: [<|im_end|>, <|endoftext|>], no sampling defaults) was added so generation stops at the end of the turn. The weights, tokenizer and chat template are unchanged.

Independent benchmark

The authors' prompts, sampling and graders (scores + scripts), vLLM bf16, seed 0. MATH-500: 500 × 4 samples. GSM8K: 1,319 × 2. IFEval: 541 × 2, strict prompt-level, official checkers, answer after </think>. Thinking on: temperature 0.6, top_p 0.95, top_k 20, presence_penalty 1.5, 32k tokens. Thinking off: temperature 0.7, top_p 0.8, 8k tokens. In parentheses: the share of answers that hit the token limit.

Table with columns: MATH-500, GSM8K, IFEval
	MATH-500	GSM8K	IFEval
Base, thinking on	7.4 (93%)	13.0 (89%)	19.2 (76%)
Edited, thinking on	50.2 (32%)	67.6 (7%)	62.1 (2%)
Authors, thinking on	7.9 → 48.1	13.8 → 67.1	21.0 → 62.3
Base, thinking off	45.8 (32%)	57.1 (4%)	52.9 (0.4%)
Edited, thinking off	43.6 (31%)	55.7 (4%)	51.2 (0.1%)

Edited minus base, paired by problem, 95% CI:

Table with columns: MATH-500, GSM8K, IFEval
	MATH-500	GSM8K	IFEval
Both thinking on (the paper's claim)	+42.8 [39.3, 46.3]	+54.7 [52.5, 56.9]	+42.9 [39.1, 46.6]
Edited thinking on vs. base thinking off (the base model's best setting)	+4.3 [1.9, 6.8]	+10.5 [8.3, 12.7]	+9.2 [5.2, 13.2]
Both thinking off	−2.2 [−4.3, −0.1]	−1.4 [−3.4, +0.6]	−1.7 [−4.2, +0.9]

All three of the authors' thinking-on results reproduce within about 2 points. The edited model with thinking on also beats the base model's best setting (thinking off) on every benchmark. With thinking off, the edit costs about 1.5–2 points, which is significant only on MATH-500.

⚠️ This removes a failure mode; it doesn't add capability. Almost all of the gain comes from answers that no longer loop to the token limit.

TODO

Independent MATH-500 benchmark
GSM8K / IFEval reproduction
ONNX / Transformers.js build
GGUF (BF16) for llama.cpp / LM Studio / Ollama
In-browser demo

Notes

The authors' grader (mathqa.py) imports an internal module (mcqa) that isn't in their release. It's only used for loop diagnostics, so the benchmark uses a one-function stand-in and leaves their file untouched.
evaluate.py also needs their internal package, so I wrote a small MATH-500 script that copies its prompt, sampling and vLLM settings.
Rebuild it yourself with exp8_math/code/apply_edit.py (base revision 2fc06364). The script checks every tensor's hash.

License

The base model is Apache 2.0 © Qwen (LICENSE). This is a modified version: 2 tensors changed. The edit is CC BY 4.0, by the authors above.

bibtex

@misc{takehi2026weightspace,  title  = {Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights},  author = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar},  year   = {2026},  url    = {https://rikiyat.github.io/blog/weight-space-programming/}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

JonathanColetti

Model Tree

Base

Qwen/Qwen3.5-0.8B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

What changed

The loop direction was subtracted from 64 output columns of mlp.down_proj in layers 15 and 19 (about 130k weights, 0.016% of the model). Everything else is byte-identical to the base model.

How to run

[!IMPORTANT] The edit only helps with thinking on. The chat template turns thinking off unless you pass enable_thinking=True. Recommended sampling (the paper's): temperature 0.6, top_p 0.95, top_k 20, presence penalty 1.5.

Table with columns: You want to run it in…, Use, Files
You want to run it in…	Use	Files
Python, GPU servers	🤗 Transformers / vLLM / SGLang	`model.safetensors-*` (bf16)
The browser (WebGPU) or Node.js	🤗 Transformers.js	`onnx/*`
Any ONNX Runtime app (C#, C++, Java, mobile…)	ONNX Runtime	`onnx/*`
A laptop / CPU / Mac, local apps	llama.cpp, LM Studio, Ollama, Jan…	`Qwen3.5-0.8B-Astra-WeightEdit-BF16.gguf`

python

from transformers import AutoModelForImageTextToText, AutoTokenizerrepo = "JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForImageTextToText.from_pretrained(repo, dtype="bfloat16", device_map="auto")msgs = [{"role": "user", "content": "What is 17 * 23? Please reason step by step, and put your final answer within \\boxed{}."}]inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True,                                 return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=4096, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

from transformers import AutoModelForImageTextToText, AutoTokenizerrepo = "JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForImageTextToText.from_pretrained(repo, dtype="bfloat16", device_map="auto")msgs = [{"role": "user", "content": "What is 17 * 23? Please reason step by step, and put your final answer within \\boxed{}."}]inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True,                                 return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=4096, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

bash

vllm serve JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit# then send chat requests with: "chat_template_kwargs": {"enable_thinking": true}

vllm serve JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit# then send chat requests with: "chat_template_kwargs": {"enable_thinking": true}

bash

npm i @huggingface/transformers@next

import { AutoProcessor, Qwen3_5ForConditionalGeneration, TextStreamer } from "@huggingface/transformers"; const repo = "JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit";const processor = await AutoProcessor.from_pretrained(repo);const model = await Qwen3_5ForConditionalGeneration.from_pretrained(repo, {  dtype: { embed_tokens: "fp16", vision_encoder: "fp16", decoder_model_merged: "fp16" },  device: "webgpu", // or "wasm" in the browser, "cpu" in Node.js (use "quantized" there)});const messages = [{ role: "user", content: "What is 17 * 23? Please reason step by step, and put your final answer within \\boxed{}." }];const text = processor.apply_chat_template(messages, { add_generation_prompt: true, enable_thinking: true });const inputs = await processor(text);await model.generate({  ...inputs, max_new_tokens: 4096, do_sample: true, temperature: 0.6, top_p: 0.95, top_k: 20,  streamer: new TextStreamer(processor.tokenizer, { skip_prompt: true, skip_special_tokens: true }),});

import { AutoProcessor, Qwen3_5ForConditionalGeneration, TextStreamer } from "@huggingface/transformers"; const repo = "JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit";const processor = await AutoProcessor.from_pretrained(repo);const model = await Qwen3_5ForConditionalGeneration.from_pretrained(repo, {  dtype: { embed_tokens: "fp16", vision_encoder: "fp16", decoder_model_merged: "fp16" },  device: "webgpu", // or "wasm" in the browser, "cpu" in Node.js (use "quantized" there)});const messages = [{ role: "user", content: "What is 17 * 23? Please reason step by step, and put your final answer within \\boxed{}." }];const text = processor.apply_chat_template(messages, { add_generation_prompt: true, enable_thinking: true });const inputs = await processor(text);await model.generate({  ...inputs, max_new_tokens: 4096, do_sample: true, temperature: 0.6, top_p: 0.95, top_k: 20,  streamer: new TextStreamer(processor.tokenizer, { skip_prompt: true, skip_special_tokens: true }),});

Which dtype to pick (each graph comes in all five; size is the language-model decoder):

Table with columns: dtype, decoder size, Best for, Agrees with bf16*
dtype	decoder size	Best for	Agrees with bf16*
`fp32`	3.0 GB	reference	100%
`fp16`	1.5 GB	WebGPU	100%
`quantized` (int8)	0.87 GB	CPU / WASM	100%
`q4f16`	0.44 GB	smallest download, WebGPU	~69%
`q4`	0.48 GB	smallest download, CPU / WASM	~69%

bash

# llama.cppllama-cli -hf JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit:BF16 \  --jinja --chat-template-kwargs '{"enable_thinking":true}' \  --temp 0.6 --top-p 0.95 --top-k 20 --presence-penalty 1.5 -c 32768 # Ollama (a version with Qwen3.5 support)ollama run hf.co/JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit:BF16

# llama.cppllama-cli -hf JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit:BF16 \  --jinja --chat-template-kwargs '{"enable_thinking":true}' \  --temp 0.6 --top-p 0.95 --top-k 20 --presence-penalty 1.5 -c 32768 # Ollama (a version with Qwen3.5 support)ollama run hf.co/JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit:BF16

LM Studio: search for JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit. Only a BF16 GGUF is provided (1.56 GB). This is text only: no mmproj vision file.

How the ONNX and GGUF files were made

Neither was re-exported. I took existing conversions of the same base revision and rewrote only the two edited tensors, so every other byte matches those well-tested builds:

ONNX: onnx-community/Qwen3.5-0.8B-ONNX-OPT @ 9a4c21d8. The down_proj tensors of layers 15 and 19 are overwritten in place in each decoder_model_merged* data file. Before writing, the script regenerates the base tensors from the bf16 weights and requires a byte-for-byte match with the shipped files in all five dtypes, including the int8/int4 block quantization. So the edited bytes are exactly what that export pipeline would have produced. embed_tokens* and vision_encoder* are unchanged copies.
GGUF: ggml-org/Qwen3.5-0.8B-GGUF Qwen3.5-0.8B-BF16.gguf @ 807dfd9a, converted from Qwen revision 2fc06364. blk.{15,19}.ffn_down.weight are swapped after checking that the base bytes match the base sha256 in astra_edit/manifest.json. The edited bytes match the edited sha256. Only general.name, general.repo_url and the base-model metadata were changed.
Checks: the ONNX fp32/fp16/int8 logits match PyTorch (mean KL ≤ 6e-4, same argmax). Transformers.js fp32 greedy output is token-for-token identical to PyTorch over 120 tokens.
Scripts: conversion/.

Independent benchmark

Table with columns: MATH-500, GSM8K, IFEval
	MATH-500	GSM8K	IFEval
Base, thinking on	7.4 (93%)	13.0 (89%)	19.2 (76%)
Edited, thinking on	50.2 (32%)	67.6 (7%)	62.1 (2%)
Authors, thinking on	7.9 → 48.1	13.8 → 67.1	21.0 → 62.3
Base, thinking off	45.8 (32%)	57.1 (4%)	52.9 (0.4%)
Edited, thinking off	43.6 (31%)	55.7 (4%)	51.2 (0.1%)

Edited minus base, paired by problem, 95% CI:

Table with columns: MATH-500, GSM8K, IFEval
	MATH-500	GSM8K	IFEval
Both thinking on (the paper's claim)	+42.8 [39.3, 46.3]	+54.7 [52.5, 56.9]	+42.9 [39.1, 46.6]
Edited thinking on vs. base thinking off (the base model's best setting)	+4.3 [1.9, 6.8]	+10.5 [8.3, 12.7]	+9.2 [5.2, 13.2]
Both thinking off	−2.2 [−4.3, −0.1]	−1.4 [−3.4, +0.6]	−1.7 [−4.2, +0.9]

⚠️ This removes a failure mode; it doesn't add capability. Almost all of the gain comes from answers that no longer loop to the token limit.

TODO

Independent MATH-500 benchmark
GSM8K / IFEval reproduction
ONNX / Transformers.js build
GGUF (BF16) for llama.cpp / LM Studio / Ollama
In-browser demo

Notes

The authors' grader (mathqa.py) imports an internal module (mcqa) that isn't in their release. It's only used for loop diagnostics, so the benchmark uses a one-function stand-in and leaves their file untouched.
evaluate.py also needs their internal package, so I wrote a small MATH-500 script that copies its prompt, sampling and vLLM settings.
Rebuild it yourself with exp8_math/code/apply_edit.py (base revision 2fc06364). The script checks every tensor's hash.

License

The base model is Apache 2.0 © Qwen (LICENSE). This is a modified version: 2 tensors changed. The edit is CC BY 4.0, by the authors above.

bibtex

@misc{takehi2026weightspace,  title  = {Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights},  author = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar},  year   = {2026},  url    = {https://rikiyat.github.io/blog/weight-space-programming/}}

@misc{takehi2026weightspace,  title  = {Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights},  author = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar},  year   = {2026},  url    = {https://rikiyat.github.io/blog/weight-space-programming/}}