Why
With thinking off, B5 still answered one kind of question from memory: whether something is still true. It
answered confidently and often stamped the answer with the date from its context, so a stale answer read as a
checked one:
Pat Gelsinger has been Intel's CEO since February 2021 and is still in the role as of today.
On the questions we wrote for this, B5 skipped the search 57 times out of 70.
Training
ORPO LoRA on B5-27B, trained on 259 pairs: the new round2 config of
Nachschlagen (54 "still ...?" pairs and 40 controls,
all B5's own output), plus round1 (165 pairs) so the earlier behavior stays anchored. Each pair is cut at the
model's first assistant turn in egirl's thinking-off context: a search
call is preferred over the answer from memory, and for the controls a direct answer over an unneeded search.
r16, alpha 32, all attention and MLP projections, lr 2e-5, beta 0.1, 2 epochs, effective batch 8; trained with
Merlina. Baked with Kessel
and checked tensor by tensor: every weight that isn't a LoRA target is identical to B5, and each target differs from
B5 by exactly alpha/r · B·A (bf16 rounding).
Numbers
Through egirl's OpenAI-compatible API with web_search, web_research, calculate and run_code, thinking off
unless noted. B5 and B6 were both run as Q8_0 GGUFs (B6 as B5 + the f16 adapter) in the same session. The
questions share no entities with the training data.
Table with columns: right first move, B5-27B, B6-27B| right first move | B5-27B | B6-27B |
|---|
| held-out (41) | 35 | 41 |
| of which "needs a search" (10) | 6 | 10 |
| held-out, thinking low | 38 | 40 |
| harder set (37) | 34 | 37 |
| of which "needs a search" (10) | 7 | 10 |
| keyed answers right (held-out / harder) | 20/20, 10/12 | 20/20, 10/12 |
The controls held: B6 answered every "no tool needed" question directly, like B5. Single runs; a gap of one or two
items is noise.
Table with columns: B5-27B, B6-27B | B5-27B | B6-27B |
|---|
| ARC-Challenge / ARC-Easy / TruthfulQA (MC, llama-perplexity) | 65.9 / 87.2 / 40.9 | 66.2 / 87.2 / 40.8 |
Not re-run for B6: Propaganda_Eval and HemBench. See B5's card
for those; the adapter is small and leaves the general benchmarks flat.
Notes
- With thinking off, searching costs time: answers to "still ...?" questions take a few seconds longer because B6 now
searches.
- egirl PR #209 adds a harness-side backstop: an answer that
dates itself to today without any tool call in the run gets one retry told to search. On B5 it rarely fired,
because most skipped searches don't carry a date; B6 doesn't need it.
- Tokenizer, chat template, vision and MTP weights are B5's, unchanged.
- Built for egirl. Other harnesses weren't a goal and weren't tested.