Data
24,000 pairs from gen_data_v2.py (seed 1977, deterministic, no LLM involved).
Eight families: arithmetic chains about objects kept in rooms, possession riddles,
tenants'-association complaints, guarded recipes, wills, lost-property ledgers,
village football scores, and auction lots. Every pair is built backwards - pick the
answer, construct the only story that forces it, decorate. Sentence order, phrasing,
names and distractors are randomized per sample so no two completions share a template.
500 held-out pairs drawn from the same pools with unseen name/object combinations.
Training
SmolLM2-360M-Instruct, full fine-tune, bf16. 1,500 steps at effective batch 32,
seq len 448, lr 2e-5 cosine with 40-step warmup, labels masked over the prompt.
Ran on a Lupine Cloud RTX PRO 6000 Blackwell in 326 seconds of step time. Final
loss around 1.75.
Held-out results
ANSWER: '59 pinches of smoked paprika'
-> The recipe for the smoked paprika sauce ... calls for exactly 59 pinches of
smoked paprika, but the recipe is a closely guarded secret. How much smoked
paprika goes into the sauce?
ANSWER: 'a rubber duck' (via ollama, sampled)
-> The estate of Petra van der Linden inventories possessions, finding only a
rubber duck among them. A formal statement confirms the presence of nothing
else. What did Petra van der Linden leave?
Not all of them land. Given "161 euros" it invented a household-budget family on
the spot and asked how much was spent on groceries, which was 100. Given "the moon"
it writes about conservatories. Arguably this makes it better at its job.
Use
With ollama (the Hub pull does not carry a system prompt, so wrap it once):
printf 'FROM hf.co/tsfrm/the-unanswerer\nSYSTEM "You are the Unanswerer. You will be given an answer. Reply with exactly one question that has this answer and that no human being has ever asked."\nPARAMETER temperature 0.75\nPARAMETER top_p 0.92\nPARAMETER min_p 0.08\nPARAMETER repeat_penalty 1.15\n' > /tmp/unanswerer.modelfile
ollama create unanswerer -f /tmp/unanswerer.modelfile
ollama run unanswerer "42"
The GGUF embeds the same system prompt in its chat template, so llama.cpp and
transformers get the behavior without any setup. Without ollama:
pip install transformers torch
python3 ask.py "42" "a colander" "Dagobert van Dongen"
Files
Table | |
|---|
| model.safetensors | 724 MB, bf16 |
| unanswerer-xl-f16.gguf | ollama-ready, F16 |
| Modelfile | system prompt + sampling for the ollama wrap |
| gen_data_v2.py | regenerates the dataset byte-for-byte |
| train_xl.py / pretok.py | reproduces the training run |
| train.jsonl / test.jsonl | included |