What's inside
A LoRA adapter (rank 16 on the attention and MLP projections: q, k, v, o, gate, up, down) and a small pointer head (256-dimensional) on google/gemma-4-E2B-it at revision 3e22461f.
Table with columns: Stage, Training data, Settings| Stage | Training data | Settings |
|---|
1. General decisions (recipes/d1a-e2b.yaml, what made E2B v0.2) | decision-v7 suite: public classification datasets plus generated policy and rule data | 2 epochs, lr 1e-4, batch 4 × accum 2, from the -it trunk; not published |
| 2. Every later D1A skill, in one run (E2B v0.6's mix) | PR labels: 4,000 English (severity-stratified), 753 Japanese, 1,500 with a blast-radius label, and 479 hermes-agent PRs created 2026-10-02 to 10-05 shown twice (once in each half); hard-v1 4,000; devtools-v1 4,000; documents-v1 2,500; JGLUE 2,000; routing: agent factory 1,216, generic 1,000; decision-v7 replay 1,500: 23,427 records | from stage 1, 1 epoch, lr 5e-5, states up to 5,120 tokens, 2,929 steps (2 h 45 m) |
Both stages ran in one Hugging Face Jobs job on one NVIDIA L4, about $3.51, finished 2026-10-12 18:33 JST. The mix is E2B v0.6's, byte for byte (sha256 efff4b16), screened before training against every evaluation partition, the 487 newer PRs below included. provenance.json has the details.
Calibration: a single temperature, T = 1.41 (90% interval [1.35, 1.48]), fitted on pooled rows of the decision-v7 calibration split and the PR-labelling development set (3,178 questions). Calibration error is 0.053 before and 0.020 after, out of fold; on the 487 newer PRs below (974 questions) it is 0.021. It is stored in d1a_config.json and head.pt, and there are no per-use-case temperatures.
Results
Pull requests newer than all training
487 hermes-agent pull requests created after every PR in the training mix, held back before training. All models are MLX 8-bit at their served temperatures (E2B-it 1.41, E2B v0.6 1.45), with the serving context. Differences are paired over PRs, with 95% bootstrap intervals. E4B v0.6 is shown for reference:
Table with columns: Question, Majority answer, E2B-it v0.6, E2B v0.6, E4B v0.6, E2B-it − E2B v0.6| Question | Majority answer | E2B-it v0.6 | E2B v0.6 | E4B v0.6 | E2B-it − E2B v0.6 |
|---|
| Severity (P0–P4) | 67.8% | 72.9% | 77.2% | 79.1% | −4.3 [−7.8, −1.0] |
| Change type | 51.5% | 87.3% | 87.5% | 90.6% | −0.2 [−3.1, +2.7] |
| P0 recall (7 PRs) | | 2/7 | 0/7 | 5/7 | |
| P1 recall (23 PRs) | | 1/23 | 0/23 | 8/23 | |
| Security type recall (14 PRs) | | 11/14 | 12/14 | 9/14 | |
It finds 3 of the 30 P0/P1 PRs where E2B v0.6 found none, but it loses more on P3 (273/330 against 295/330) and P2 (74/121 against 77/121) than it gains.
Every other suite
Not measured yet. D1A's card-suite gate (transfer-v4, hard-v1, devtools-v1, documents-v1, decision-v7, JGLUE) has not run on this build. This card will not quote numbers for them until it has.
Photos, voice and video
With the media extra, d1a.serving.serve answers questions about a photo, a voice clip or a video through Gemma 4 E2B-it's own vision and audio encoders, unchanged by the adapter (POST /v1/systemone/media). No media in training (zero-shot), measured on only the playground's 14 photo, voice and video demo requests. It answers all 14:
- 21 of 28 answers match E4B v0.6 (E2B v0.6: 25 of 28). It misses damage E4B sees (the crushed-truck photo and video, the wet-parcel video), calls two voice clips urgent (the English directions and the Japanese not-at-home message), and places the damaged-door and intact-mailbox photos "on a sidewalk".
- 19 of 28 match E2B v0.6. Unlike E2B v0.6, it calls the intact locker and mailbox photos intact.
The questions it was trained for
PR labelling works best with these questions, worded exactly so, over a document of the form title / author / stats / body / files (one - status path +added/-deleted line per file):
type (choice): "Primary change type from files and body, not the title prefix." Options: type/bug, type/docs, type/feature, type/perf, type/refactor, type/security, type/test.
blast (choice): "How far a mistake in this PR spreads in production." Options: review:blast-contained (one module), -moderate (one subsystem), -broad (shared helper or config), -massive (auth, permissions, or all paths).
sev (choice): "How serious the problem this PR addresses is — not the risk of merging the diff as-is." Options P0 to P4.
The PR-labeler recipe has the exact wording, the document builder and the training and scoring scripts.
Run it
Needs d1a with jonpol01/d1a#261 (the <bos> above):
pip install "d1a[serve] @ git+https://github.com/jonpol01/d1a"python -m d1a.serving.serve --run JohnP1/d1a-e2b-it@v0.6 --port 8009
or in-process: from d1a import D1A; D1A.load("JohnP1/d1a-e2b-it@v0.6").decide(state, questions). Builds of the same model: MLX 8-bit JohnP1/d1a-e2b-it-mlx-q8 (with the photo, voice and video encoders), MLX 4-bit JohnP1/d1a-e2b-it-mlx-q4, Core ML JohnP1/d1a-e2b-it-coreml (int8) and JohnP1/d1a-e2b-it-coreml-4bit (int4). Unless your app already loads Gemma 4 E2B-it, use JohnP1/d1a-e2b v0.6 instead.
Limits
- Experimental, and below E2B v0.6 where measured: severity −4.3 points on newer PRs, media 21/28 against 25/28 agreement with E4B v0.6. Its pre-registered bar failed (above).
- Not a PR labeller. E4B v0.6 remains the PR labeler. On the 487 newer PRs this build finds 3 of 30 P0/P1 PRs (E4B v0.6: 13 of 30). Use E4B for severity, or flag a PR when p(P0) + p(P1) is high instead of taking the top answer.
- Most suites are not measured: hard, developer-tool and long-document decisions, held-out transfer, general decisions (decision-v7) and Japanese (JGLUE). Treat its answers there as unknown quality.
- Gate tool calls through D1A's fail-safe thresholds (
d1a.agents.presets.advise: deny at p ≥ 0.5, allow only at p ≥ 0.8, otherwise ask), never through the raw top answer. E2B v0.6 gave a near-tie on the playground's $4,999 payment example; this build's demo answers have not been read yet.
<bos>: requests encoded without the leading id 2 are out of distribution (see above).
- Media: zero-shot, measured on 14 demo requests only. It misses damage on 3 of them (above).
- One temperature for every question, and no use-case temperatures.
- The recent PR labels come from one large open-source project and a few days of it. If its conventions move again, so will these numbers.
- Questions are in English. The documents can be English or Japanese (trained on both; Japanese not measured on this build).
Versions
Table with columns: Tag, What it is| Tag | What it is |
|---|
v0.6 | E2B v0.6's recipe on the Gemma 4 E2B-it trunk; experimental, Bar B of #227 failed (severity −4.3, media 21/28) |
License and data
Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: github.com/jonpol01/d1a, built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.