About OpenMed
OpenMed builds open models and open-source tools for clinical and biomedical text that run on your own hardware.
Results
Same model architecture, tokenizer, chat template and sampling for every row; only weight values differ. One sample per
question with seed 0 (MedQA also with seeds 1 and 2). Thinking on, temperature 0.6, top_p 0.95, top_k 20, presence
penalty 1.5, at most 32,768 new tokens (the paper's evaluation protocol). The prompt is OpenMed's fixed MedQA exam prompt
(below), and an answer is graded from the text after </think>: its last Answer: X line, X in A to D. Answers cut at the
token limit and answers with no gradable line count as wrong, as in the paper.
Table with columns: Checkpoint, MedQA (seed 0), MedQA, median of 3 seeds (range), MedMCQA, MMLU medical, MedQA answers cut at 32,768 tokens, MedQA answers with no gradable line| Checkpoint | MedQA (seed 0) | MedQA, median of 3 seeds (range) | MedMCQA | MMLU medical | MedQA answers cut at 32,768 tokens | MedQA answers with no gradable line |
|---|
| OpenMed WSP Math + Clinical 0.8B (this model) | 45.4% | 45.4% (44.1% to 47.0%) | 38.8% | 60.2% | 0.2% | 0.3% |
| Qwen3.5-0.8B + the paper's math edit (before OpenMed's clinical edit) | 32.9% | 32.9% (32.8% to 33.6%) | 33.3% | 52.4% | 0.1% | 28.8% |
| Random patch, same size, random direction (seed 1) | 32.4% | n/a | 33.7% | 50.3% | 0.1% | 31.7% |
| Random patch, same size, random direction (seed 2) | 34.0% | n/a | 33.1% | 50.9% | 0.1% | 29.1% |
| Qwen3.5-0.8B as released | 24.4% | 23.1% (22.1% to 24.4%) | 30.6% | 43.4% | 18.3% | 32.9% |
Paired tests on the same questions (seed 0; exact McNemar on the questions only one checkpoint got right; bootstrap 95%
CI over questions, 10,000 resamples):
Table with columns: Comparison, Benchmark, n, Before, After, Difference (95% CI), Only before right / only after right, McNemar p| Comparison | Benchmark | n | Before | After | Difference (95% CI) | Only before right / only after right | McNemar p |
|---|
| Qwen3.5-0.8B -> WSP Math + Clinical 0.8B | MedQA-USMLE (4 options) | 1,273 | 24.4% | 45.4% | +21.0 pts (+18.1 to +23.8) | 72 / 339 | 1.6e-42 |
| Qwen3.5-0.8B -> WSP Math + Clinical 0.8B | MedMCQA | 1,000 | 30.6% | 38.8% | +8.2 pts (+5.0 to +11.4) | 98 / 180 | 1.0e-06 |
| Qwen3.5-0.8B -> WSP Math + Clinical 0.8B | MMLU | 1,089 | 43.4% | 60.2% | +16.7 pts (+13.6 to +19.8) | 72 / 254 | 6.1e-25 |
| + math edit -> WSP Math + Clinical 0.8B | MedQA-USMLE (4 options) | 1,273 | 32.9% | 45.4% | +12.5 pts (+9.7 to +15.3) | 102 / 261 | 3.4e-17 |
| + math edit -> WSP Math + Clinical 0.8B | MedMCQA | 1,000 | 33.3% | 38.8% | +5.5 pts (+2.4 to +8.5) | 98 / 153 | 6.2e-04 |
| + math edit -> WSP Math + Clinical 0.8B | MMLU | 1,089 | 52.4% | 60.2% | +7.7 pts (+5.0 to +10.5) | 76 / 160 | 4.8e-08 |
| Qwen3.5-0.8B -> + math edit | MedQA-USMLE (4 options) | 1,273 | 24.4% | 32.9% | +8.5 pts (+5.7 to +11.2) | 115 / 223 | 4.4e-09 |
| Qwen3.5-0.8B -> + math edit | MedMCQA | 1,000 | 30.6% | 33.3% | +2.7 pts (-0.5 to +5.8) | 116 / 143 | 0.106 |
| Qwen3.5-0.8B -> + math edit | MMLU | 1,089 | 43.4% | 52.4% | +9.0 pts (+6.0 to +11.9) | 101 / 199 | 1.6e-08 |
| random patch 1 -> WSP Math + Clinical 0.8B | MedQA-USMLE (4 options) | 1,273 | 32.4% | 45.4% | +13.0 pts (+10.2 to +15.9) | 98 / 264 | 1.0e-18 |
| random patch 1 -> WSP Math + Clinical 0.8B | MedMCQA | 1,000 | 33.7% | 38.8% | +5.1 pts (+2.0 to +8.1) | 99 / 150 | 0.001 |
| random patch 1 -> WSP Math + Clinical 0.8B | MMLU | 1,089 | 50.3% | 60.2% | +9.8 pts (+7.1 to +12.6) | 70 / 177 | 7.2e-12 |
| random patch 2 -> WSP Math + Clinical 0.8B | MedQA-USMLE (4 options) | 1,273 | 34.0% | 45.4% | +11.4 pts (+8.6 to +14.3) | 108 / 253 | 1.5e-14 |
| random patch 2 -> WSP Math + Clinical 0.8B | MedMCQA | 1,000 | 33.1% | 38.8% | +5.7 pts (+2.6 to +8.7) | 97 / 154 | 3.9e-04 |
| random patch 2 -> WSP Math + Clinical 0.8B | MMLU | 1,089 | 50.9% | 60.2% | +9.3 pts (+6.4 to +12.0) | 77 / 178 | 2.2e-10 |
Also accepting Answer: <B>, Answer: (b), Answer: <letter>B and \boxed{B} after </think>. The gap between this
and the main score shows how much of a checkpoint's score is lost to the format of its final line rather than to the
choice itself.
Table with columns: Checkpoint, MedQA, MedMCQA, MMLU medical| Checkpoint | MedQA | MedMCQA | MMLU medical |
|---|
| OpenMed WSP Math + Clinical 0.8B (this model) | 45.4% | 38.8% | 60.2% |
| Qwen3.5-0.8B + the paper's math edit (before OpenMed's clinical edit) | 39.4% | 37.5% | 56.4% |
| Random patch, same size, random direction (seed 1) | 39.8% | 37.5% | 55.7% |
| Random patch, same size, random direction (seed 2) | 39.5% | 35.8% | 55.7% |
| Qwen3.5-0.8B as released | 32.5% | 34.8% | 50.0% |
Nothing else broke (general benchmarks, one sample, seed 0)
Table with columns: Checkpoint, MATH-500, MATH-500 cut at 32,768 tokens, IFEval (strict, prompt level)| Checkpoint | MATH-500 | MATH-500 cut at 32,768 tokens | IFEval (strict, prompt level) |
|---|
| OpenMed WSP Math + Clinical 0.8B (this model) | 51.2% | 31.0% | 62.5% |
| Qwen3.5-0.8B + the paper's math edit (before OpenMed's clinical edit) | 50.6% | 30.2% | 61.4% |
| Qwen3.5-0.8B as released | 7.4% | 93.6% | 20.2% |
The paper reports 48.1% on MATH-500 and 62.3% on IFEval for its math edit (4 and 2 samples per problem).
What the edit does
The fixed exam prompt ends with "finish with one last line exactly like: Answer: ". Under the paper's sampling the model often copies the placeholder instead of filling it in, ending with Answer: <letter> or Answer: <B>, which no grader reads as a letter. The editor traced this to attention head 4 of layer 23, the last layer: at the final Answer: position it attends to the prompt's < and writes the direction that raises the logit of the < token. The presence penalty makes it worse, because the four letters have usually appeared in the thinking already while < has not.
The edit removes that one direction from the head's output: with u the unit vector of (final-norm weight x embedding of <), the 256 columns of o_proj that belong to head 4 become W - u u^T W. Nothing else in the model changes. On 2,560 practice questions from the training splits (ten fresh sets, never used to design the edit) the editor measured a paired gain of +9.65 +/- 1.01 points with the residual ungradable answers down to 0.3%.
Table with columns: Tensor, Shape, bfloat16 entries changed, Relative change, sha256 (edited, bfloat16 bytes)| Tensor | Shape | bfloat16 entries changed | Relative change | sha256 (edited, bfloat16 bytes) |
|---|
model.language_model.layers.23.self_attn.o_proj.weight | 1024 x 2048 | 234,183 | 0.0333 | 8fac195c18d739da8a38cc2b56577813617655d145912bd2352dce9d750f0951 |
234,183 bfloat16 entries differ from the starting checkpoint (the math-edited checkpoint ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming @ f4a5dd43, whose two edited tensors match the paper authors' published diff bit for bit). edit/manifest.json and
edit/edit.npz hold the edit as an exact diff (indices and bfloat16 bits), with the sha256 of every tensor before and
after; rebuilding the tensors from the diff reproduces these hashes.
Quick start
from vllm import LLM, SamplingParamsfrom transformers import AutoTokenizer def main(): repo = "OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical" tok = AutoTokenizer.from_pretrained(repo) llm = LLM(model=repo, dtype="bfloat16", max_model_len=34816, language_model_only=True, generation_config="vllm") # a MedQA-USMLE practice question (train split, id medqa_train_4541; the key is B) question = ("A 23-year-old male with a homozygous CCR5 mutation is found to be immune to HIV infection. " "The patient's CCR5 mutation interferes with the function of which viral protein?") options = {"A": "Reverse transcriptase", "B": "gp120", "C": "gp41", "D": "pp17"} user = ("Which option is the best answer to this clinical question?\n\n" + question + "\n\n" + "\n".join(f"{k}. {v}" for k, v in options.items()) + "\n\nReason as much as you need, then finish with one last line exactly like: Answer: <letter>") prompt = tok.apply_chat_template([{"role": "user", "content": user}], tokenize=False, add_generation_prompt=True, enable_thinking=True) sp = SamplingParams(temperature=0.6, top_p=0.95, top_k=20, presence_penalty=1.5, max_tokens=32768, seed=0) text = llm.generate([prompt], sp)[0].outputs[0].text print(text.split("</think>")[-1].strip()) if __name__ == "__main__": # vLLM starts worker processes main()
How it was made
- The paper's math edit, verified. We started from Hugging Face's ML Intern reproduction of the paper's 50-pass edit and checked that its two edited tensors are byte-identical to the authors' published diff (
RikiyaT/lm_understand-logs, exp8_math/runs/p50c/edit/) and that every other tensor equals Qwen/Qwen3.5-0.8B. On medicine it already helps (table above): the thinking loops that hit the 32,768-token limit almost disappear.
- The clinical edit. We rebuilt the paper's harness, whose prompts and rules are published but whose code is not: a persistent CPU Python kernel over the weights (no GPU, no internet, no gradients;
save_edit refuses the final norm, the embeddings and whole-tensor rescaling) and a forward-pass tool (generations with the evaluation's sampling, paired original vs edited with fixed seeds, teacher-forced reads of hidden states, MLP activations and attention). The task text is the paper's, with the target changed from math to clinical questions. The editor could use only training-split practice questions; the MedQA, MedMCQA and MMLU test sets above were never shown to it. The editor is a frontier LLM. The run was stopped after 21 of its 50 forward passes; the editor had settled on this edit by its 6th pass and spent the rest confirming it.
- Evaluation. Every checkpoint ran with the same vLLM 0.33.0 server and settings on one GPU. The random-patch rows apply a change of the same size to the same 256 columns of the math-edited model (same singular values, random directions; two seeds): the direction matters, not the size of the change.
Files
- The weights, config, tokenizer and chat template: Qwen3.5-0.8B's release files with the edited tensors (every
other tensor is unchanged).
edit/: the clinical edit as an exact, hash-checked diff from the starting checkpoint.
eval/: per-question results of every checkpoint on every benchmark (id, finish reason, tokens, extracted answer,
grade) with no question text, the scores, the first 20 generated texts per benchmark for this model, and
receipt.json (forward passes, Python runs, turns and wall time of the edit run).
Limitations
- One 0.8B model and one sample per question (MedQA with three seeds). Gains are relative to the same model and sampling; they say nothing about clinical safety.
- Most of the gain is the model now stating a letter where it used to copy the placeholder; the format-agnostic table shows how much of it is format and how much is choice.
- The fixed prompt literally says
Answer: <letter>; with a prompt that does not show a placeholder the clinical edit has less to fix.
- The edit lowers the probability of
< everywhere, not only in the answer line (the editor measured a median change of 0.0008 nats per thinking token).
- Research only. Not for clinical use, diagnosis or treatment.
Credits
- Method: "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights" by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL, 2 Nov 2026), https://rikiyat.github.io/blog/weight-space-programming/. Their logs and diffs: https://github.com/RikiyaT/lm_understand-logs (logs, data and networks CC BY 4.0, scripts MIT). The math edit inside this model is theirs (p50c), used under CC BY 4.0.
- Reproduction we built on: Hugging Face's ML Intern rebuilt and verified the math edit (ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming). ML Intern did not take part in OpenMed's clinical edit or evaluation.
- Base model: Qwen/Qwen3.5-0.8B by the Qwen team, Apache-2.0.
- Data: MedQA-USMLE (GBaker/MedQA-USMLE-4-options, CC-BY-4.0), MedMCQA (openlifescienceai/medmcqa, Apache-2.0), MMLU (cais/mmlu, MIT), MATH-500 (HuggingFaceH4/MATH-500), IFEval (google/IFEval, Apache-2.0).
Citation
@misc{takehi2026weightspace, title = {Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights}, author = {Rikiya Takehi and Jayden Teoh and Alex Zhang and Omar Khattab}, year = {2026}, month = {nov}, url = {https://rikiyat.github.io/blog/weight-space-programming/}}