Benchmark
Test set: 249 functions from source files that appear nowhere in training (the split is grouped by source file).
Every candidate is judged by fnvcheck, the project's own checker: compile with the real MSVC 2008 compiler, compare
with the original function byte for byte. A function only counts as solved on a byte-exact MATCH that also follows
the project rule against filler arrays. Base and fine-tuned model ran with the same prompts, the same token budget
(min(4096, 1024 + 8 x size) new tokens) and the same llama.cpp Q8_0 runtime; 1 greedy + 8 sampled (T = 0.7, top-p 0.95)
completions per function. The test set was used once for this comparison; checkpoints were chosen on a separate dev set.
Table with columns: Metric (249 unseen functions), Base Qwen3.5-4B, This model| Metric (249 unseen functions) | Base Qwen3.5-4B | This model |
|---|
| Greedy byte-exact match | 0.4% | 18.5% |
| Greedy match, excluding 20 functions that near-duplicate a training example ("novel") | 0.0% | 13.1% |
| pass@1 (sampled) | 0.4% | 14.9% |
| pass@4 (sampled) | 1.3% | 19.7% |
| pass@8 (sampled) | 2.4% | 21.3% |
| Solved by any of the 9 completions | 2.4% | 22.5% |
| Greedy output compiles | 10.0% | 51.8% |
Paired comparison on the same 249 functions: 46 solved only by the fine-tune, 1 only by the base model
(exact McNemar p = 6.8e-13); sampled pass@1 difference +14.6 points (bootstrap 95% CI +10.7 to +18.6).
All 46 greedy matches were re-checked independently on freshly built packs, and none appears verbatim in any training answer.
Greedy match by slice (fine-tune / base):
Table with columns: Slice, n, This model, Base| Slice | n | This model | Base |
|---|
| Function size < 16 bytes | 44 | 68.2% | 2.3% |
| 16-63 bytes | 57 | 24.6% | 0.0% |
| 64-255 bytes | 131 | 1.5% | 0.0% |
| 256-1023 bytes | 12 | 0.0% | 0.0% |
| >= 1024 bytes | 5 | 0.0% | 0.0% |
Built with /Od | 84 | 20.2% | 1.2% |
Built with /O2 | 165 | 17.6% | 0.0% |
Honest summary: the model has learned small, patterned functions well and is useful there; it rarely solves
functions of 64 bytes or more. In a live run against the project's open queue (121 claimed functions, almost all
128-255 bytes, every one previously attempted by other agents) it produced no matches. fnvcheck's --solve-slots
and --permute near-miss solvers rescued only 3 of 705 near misses.
Use the Qwen chat template with thinking disabled (enable_thinking=False). The model was trained on exactly this
format; other formats will do worse.
System message:
You are a matching decompiler for Fallout: New Vegas (PC, 1.4.0.525), built with Microsoft Visual C++ 2008 SP1 (cl 15.00.30729.01).Given one function's machine code and the skeleton of the source file it belongs to, write C++ that this compiler turns into exactly the same bytes.Rules: real, readable C++ only; no inline assembly, naked functions or raw bytes; no hard-coded absolute addresses. Name the function exactly as given. Declare any stand-in types, fields (unkXX at their offsets, never filler arrays) and callees you need; use names the skeleton already declares.Answer with one ```cpp block holding the complete candidate and nothing else.
User message (placeholders in braces):
Function: {symbol, e.g. FUN_00xxxxxx}Size: {size} bytesCompiler flags: {flags, e.g. /MT /O2 /GS /EHsc /Gy} Target machine code:```asm{the function's listing as rendered by fnvcheck: fc_common.render_function(code, addr, size, Namer())}``` Source file skeleton ({unit file name}[, excerpt]):```cpp{the context pack's unit file; if longer than ~9000 characters, the #include lines plus the blocks that mention the most FUN_/DAT_/str_ names from the listing, in file order, gaps marked "// ..."}```
The listing must come from your own legitimately purchased copy of the game, rendered with fnvcheck
(available to fnvdecomp contributors). No game code, machine code or disassembly is distributed here.
Recommended decoding: greedy first; stop at the closing "```" of the code block; if a completion runs into the token
limit it has entered a repetition loop - discard it and sample again. Always verify with fnvcheck before submitting
anything to fnvdecomp.
Usage
llama.cpp (as benchmarked; Vulkan on an AMD Strix Halo iGPU reached ~45 tok/s single-stream, ~350 tok/s at 32 parallel):
llama-server -m qwen3.5-4b-fnvdecomp-r1-q8_0.gguf -ngl 99 -fa on -c 16384
Send the prompt already rendered by the Qwen chat template (enable_thinking=False) to /completion.
Transformers + PEFT (load the text-only causal LM):
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B")model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "ortexvortex/Qwen3.5-4B-fnvdecomp-r1")prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
Training
- Data: 3,330 (prompt, verified source) pairs over 2,733 functions. Sources were contributor matches that were
re-verified byte-exact with fnvcheck on the training machine, source by source (99.95% of contributor claims that
could be compiled reproduced). Sources using filler arrays were removed. Prompts contain the target's real machine
code, rendered locally from the game image. The training data is not released (project rules forbid sharing machine
code or disassembly).
- Split: grouped by source file; 2,959 / 86 / 249 functions train / dev / test, no file shared across splits.
- Method: LoRA r = 64, alpha = 64, no dropout, on every language-model projection - full attention (q/k/v/o), the
Gated-DeltaNet linear attention used in 24 of 32 layers (in_proj_qkv, in_proj_z, out_proj) and the MLP. Loss on the
answer tokens only. 2 epochs, 616 steps of ~32k tokens, AdamW, lr 1e-4 cosine with 5% warmup, bf16 autocast with
fp32 LoRA weights (and the checkpoint's fp32 linear-attention norms kept in fp32).
- Checkpoint: the final one (step 616), chosen on dev greedy match rate; dev loss 0.487 (base) -> 0.189.
- Hardware: one AMD Ryzen AI Max+ 395 (Radeon 8060S iGPU, 128 GB unified memory), PyTorch ROCm; ~11.6 h.
A second run with 22% more data (r2) tied with this one on test (16.5% greedy, McNemar p = 0.23), so r1 is published.
Limitations
- Useful mainly for functions under 64 bytes; mid-size and large functions are rarely solved.
- About 1 in 5 sampled completions on harder functions degenerates into a repetition loop.
- Trained only on Fallout: New Vegas 1.4.0.525 with MSVC 2008 SP1 and fnvdecomp's prompt format; it is not a
general-purpose decompiler.
- Output must always be verified with fnvcheck; follow fnvdecomp's rules
before submitting anything.