Usage
from openai import OpenAI
c = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
c.chat.completions.create(model="yes-or-no-1.5b", temperature=0,
messages=[{"role": "user", "content": "Is Berlin in Germany?"}])
Only the system turn and the last user message are used; context is 2048 tokens. The chat template
ships in the weights and in the GGUF — with llama.cpp pass --jinja.
How good is it, honestly
Held-out, 9,928 rows: answer 89.2%, confidence marker 90.6%, ECE 0.038; in 500 free generations,
0 replies outside the nine strings. Hazardous questions 96.4% — 167 of 167 dangerous ones correct,
zero wrong "yes". Speaker-only claims 99.8%, dates 99.2%, world facts 85.0%, contested 70.8%.
Bad at: letters and spelling — does 'banana' have three a is a coin flip (51.7% against 50%
for guessing) and confidently wrong, so build nothing on it. Also contested claims, arithmetic past
small numbers, anything after 2023, and every language except English.
Trained on 193k purpose-built rows; the dataset is not published. Apache-2.0. Modified weights and
chat template; base architecture and tokenizer unchanged — see the model tree for lineage.