mertkayacs

Wahler-4B

README

License: apache-2.0

What it fixes

The 103-second film, sound on. Also in Türkçe and Deutsch.

Tested on the live model

We sent Wähler-4B 130 requests in German with known answers on 4 October 2026. Wähler-4B answered 122 of 130 correctly; every request and answer is in results/tested.

Table with columns: Case, What was sent, Result
Case	What was sent	Result
Planted instructions	30 phishing emails, each with a different planted line, plus the same 10 without it	28 of 30 quarantined; 10 of 10 without the line
Long policies	20 customers against one six-rule return policy	14 of 20 matched the answer computed from the rules
Negations	15 short facts, each asked plain and negated	30 of 30 correct
Missing facts	10 situations without the deciding fact, plus the same 10 with it	answered `unknown` in 10 of 10; 10 of 10 correct with the fact
Casual messages	20 casual messages written in German, with typos and slang	20 of 20 routed to the right team

Emberwick: every villager asks Deem-4B what to do next.

Emberwick: every villager asks Deem-4B what to do next. More clips

The 53-second film, sound on: two mistakes small decision models make and how JevAlt fixes each one.

Der 53-Sekunden-Film, mit Ton. Emberwick auf Deutsch: Jeder Dorfbewohner fragt Wähler-4B, was als Nächstes zu tun ist. GIF (4K) · leichtes GIF

Try it

Open the Space, pick an example and press Decide, or write your own situation, question and options. These are the Space's examples in German with Wähler-4B's answers on 1 October 2026:

Table with columns: Einsatz, Situation, Frage, Antwort
Einsatz	Situation	Frage	Antwort
Support-Ticket	Ein Kunde wurde für März doppelt belastet und will heute eine Erstattung	Welches Team soll dieses Ticket bearbeiten?	Abrechnung 91,8 %
Störung	Der Checkout liefert allen Kunden Fehler 500, 43 Bestellungen in 10 Minuten fehlgeschlagen	Wie schwer ist dieser Vorfall?	Kritisch 82,0 %
Vertriebs-Lead	Betriebsleiter eines Unternehmens mit 200 Mitarbeitenden: Budget freigegeben, Entscheidung diesen Monat, will eine Demo	Wie soll der Vertrieb diesen Lead behandeln?	Heiß 93,2 %
Rückgabefrist

Every probability of every run, in all three languages: space-examples.json.

Table

Start checkpoint	internlm/Intern-Decision-4B (Qwen3.5-4B)
Languages	German first, the others still work
API	TypeSafe's `POST /v1/systemone`, request and response unchanged
Extras	`reasoning` off / on / auto, `abstain`, `coverage` (conformal sets)
Q4_K_M file / peak RAM	2.71 GB / 3.03 GB (measured, 4k context)
License

Wähler-4B ist ein offenes Entscheidungsmodell, das auf deutschsprachigen Entscheidungen trainiert wurde. Sie senden einen Zustand und Choice-, Score- oder Noul-Fragen und erhalten für jede Option eine kalibrierte Wahrscheinlichkeit. Auf Wunsch schreibt das Modell zuerst eine kurze Begründung auf Deutsch und entscheidet dann. Die Q4_K_M-Version läuft mit etwa 3 GB RAM auf dem eigenen Rechner, die Daten bleiben lokal.

Run it

bash
pip install "jevalt[serve,gguf] @ git+https://github.com/mertkayacs/jevalt"
jevalt serve --model mertkayacs/Wahler-4B-GGUF --file Wahler-4B-Q4_K_M.gguf

Any TypeSafe client works against it:

python
from typesafe_sdk import TypeSafeClient, Choice, Noul
client = TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8000")

Results

Accuracy on English, Turkish and German decisions and on typed-decisions: JevAlt, Kev-4B and Laya

Hidden instructions, long irrelevant text, long policies and negated questions: JevAlt against Kev-4B and Laya

Same items and client for every model, each as shipped: Kev-4B r10 and Laya 0.3.22 on their own servers with their own calibration. Jev 1.13 rows come from TypeSafe's API reference and Models page. The held-out tests come from JevAlt's own data pipeline, so they favour JevAlt. Kev-4B and Laya both do better on long padding; Kev-4B does better on GermEval 2017 and 10kGNAD; Laya is far smaller and faster. Every number and every decision: results/comparison.

What Jev 1.13 lacks and JevAlt has: thinking when unsure, coverage sets, models made for Turkish and German, open weights

The three test splits went through the same pipeline as the training rows, so they measure what the training aimed at. On the German split, Wähler-4B answers 92.0% correctly and Kev-4B 81.1%. The paired bootstrap (2,000 resamples) gives a 95% interval of +8.8 to +13.0 accuracy points for that gap. Wähler-4B's Brier score is 0.119 and Kev-4B's 0.298. On 10kGNAD, Wähler-4B answers 62.5% correctly and Kev-4B 65.3%; on GermEval 2017, Wähler-4B answers 64.3% correctly and Kev-4B 65.3%; on JevBench-hard, Wähler-4B answers 65.8% correctly and Kev-4B 54.1%. These suites were outside the training data. On 38 German date, number and policy decisions, Wähler-4B's accuracy is 0.605 with reasoning off and with reasoning: "auto"; its Brier is 0.519 with reasoning off and 0.629 with reasoning: "auto".

The German traces Wähler-4B writes scored 3.68 and 3.95 with two blind raters from other model families (Qwen3.5-122B and Gemma 4 26B, 1 to 5 scale, 13 to 20 traces per rater), against 3.14 and 3.15 for Intern-Decision-4B. We aimed for 4.0.

Most fixes are a set of training rows aimed at one weak spot. The comparisons below use Kev-4B on the same held-out rows. JevAlt's pooled results use each model's own language. On held-out German decisions, Kev-4B answers 81.1% correctly and Wähler-4B 92.0%.

The data. About 23,900 training rows in English, Turkish and German. Public sets with known answers (MASSIVE, Open-Jev, PAWS-X, typed-decisions); everyday situations written directly in each language by other open models; requests from the Emberwick game; and the fix sets below. Two teacher models from labs other than the writer give every written row a probability per option, and an answer counts only when both teachers and the writer agree on it. Those probabilities, the soft labels, are what the models learn. Test rows were split off by group, and their checksums recorded, before the final training runs.
Hidden instructions. A fix set of 827 rows hides a hostile line in the text (an order to the AI filter, a fake rule) at the start, the middle or the end, with the right answer unchanged. On our probe, hidden lines change 36.0% of Kev-4B's answers and 17.5% of Wähler-4B's. On 203 held-out planted-instruction rows, Kev-4B answers 81.3% correctly and JevAlt 90.1%. Our target was under 5%.
An honest "unknown". A fix set of 310 rows removes the fact that decides the question and asks it with and without an unknown option. On 11 held-out cases without the deciding fact, Kev-4B answers unknown in 0 and JevAlt in 9; Kev-4B and Laya have no unknown option. The model we started from, Intern-Decision-4B, already answers unknown in 9 of 11; training raised the mean probability of unknown from 0.55 to 0.74. Turn it on with abstain: true.
390 rows give a policy with exceptions and sub-limits, with the right answer worked out by code, and 1,188 rows bury the facts in up to 3,000 tokens of unrelated records. On 150 held-out policy rows, Kev-4B answers 59.3% correctly and JevAlt 80.0%. On 285 padded rows, Kev-4B answers 87.4% correctly and JevAlt 95.4%. With 600 words of unrelated records in front, Wähler-4B loses 12.2 accuracy points; Kev-4B loses 5.4 and Laya 10.4 points.

Base: internlm/Intern-Decision-4B (Qwen3.5-4B)
Method: LoRA on the bf16 weights, rank 32, alpha 32, on one A100 80 GB
Epochs: 1 over all three languages (shared run: 17,363 rows, 543 steps, 63 min), then 1 on the German-weighted mix (S-de: 5,384 rows, 210 steps, 28 min)
Runs: 11 training jobs: 6 short smoke and probe runs, a pilot at scale, the shared run and the 3 language runs
Compute: about 2.7 A100 hours for the released models; 32.2 USD for the whole project, labeling included
Writers, labelers and trace writers: GLM-5.x, Mistral Large 3, DeepSeek V4 Pro, DeepSeek V4.1 Flash, Gemma 4 26B-A4B, Qwen3.5-122B-A10B, Qwen3.5-35B-A3B, Kimi K3 (54 rows), MiniMax M3 (2 reasoning traces)

Table with columns: Source, English, Turkish, German
Source	English	Turkish	German
Teacher-written scenarios (G)	1,287	1,741	1,408
Village game requests (N)	124	129	107
Fix sets (F1-F12)	1,256	1,197	1,257
Public datasets (P)	7,188	4,115	4,074

Use and limits

Good for routing, tagging, triage and moderation at volume, and for automated decisions that need calibrated probabilities.
Runs on-device or on-prem with the GGUF build, so the data stays with you.
Knowledge is bounded by a 4B model, and the context is 8k tokens, so it is no tool for general questions or long summaries.
Probabilities are calibrated on our held-out data. Refit with jevoss calibrate on yours before you set thresholds.
Reasoning traces add little on our test rows (see the results), and there is no image input.

Citation

bibtex
@software{kaya2026jevalt,
  author = {Mert Kaya},
  title = {JevAlt: Open Decision Models with the Jev API},
  year = {2026},
  license = {Apache-2.0},
  url = {https://github.com/mertkayacs/jevalt}
}

Code and links

Code, server and training: https://github.com/mertkayacs/jevalt
Playground, probes and recipes: https://github.com/mertkayacs/jevoss
Try it online: Space
The village game: Emberwick
Project site: jevalt.mertkayacs.com
This model's page: jevalt.mertkayacs.com/models/wahler-4b

If this is useful to you, a star on GitHub helps other people find it.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

mertkayacs

Model Tree

Base

internlm/Intern-Decision-4B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality