D1A models: what each can do (2026-10-17)
Table with columns: D1A-E4B-it v0.6, D1A-E4B v0.6.1, D1A-E2B v0.6.1, D1A-E2B-it v0.6 | D1A-E4B-it v0.6 | D1A-E4B v0.6.1 | D1A-E2B v0.6.1 | D1A-E2B-it v0.6 |
|---|
| Base model (trunk) | Gemma 4 E4B-it (instruction-tuned) | Gemma 4 E4B (pretrained base) | Gemma 4 E2B (pretrained base) | Gemma 4 E2B-it (instruction-tuned) |
Decisions (/v1/systemone) | yes | yes | yes | yes (experimental) |
Photos, voice and video (MLX builds with media/) | yes, zero-shot | yes, zero-shot | yes | yes |
| Chat / text generation | yes: unmerged build, adapter off = Gemma 4 E4B-it (Enable chat) | no | no | yes (same way) |
| Embeddings | from a GGUF embedder served beside it (D1A_EMBED_GGUF) | — | — | same |
| Routing temperature | no (measured, not shipped) | yes, T 1.18 | yes, T 0.95 | no |
| Calibration temperature | 1.55 | 1.95 | 1.45 | 1.41 |
| Builds | PyTorch adapter JohnP1/d1a-e4b-it; MLX -mlx-q8 (7.0 GB with media) and -mlx-q4 (4.9 GB with media), both merged | PyTorch, MLX 8-bit, Core ML int8 preview | PyTorch, MLX 8-bit, Core ML int8 | PyTorch adapter, MLX q8/q4, Core ML int8/int4 |
Kept current in the D1A repository: docs/MODELS.md.
What's inside
A LoRA adapter (rank 16 on the attention and MLP projections: q, k, v, o, gate, up, down) and a small pointer head (256-dimensional) on google/gemma-4-E4B-it at revision ee0ef602.
Table with columns: Stage, Training data, Settings| Stage | Training data | Settings |
|---|
1. General decisions (recipes/d1a-e2b.yaml, what made E2B v0.2) | decision-v7 suite: public classification datasets plus generated policy and rule data | 2 epochs, lr 1e-4, batch 4 × accum 2, from the -it trunk; not published |
| 2. Every later D1A skill, in one run (E2B v0.6's mix) | PR labels: 4,000 English (severity-stratified), 753 Japanese, 1,500 with a blast-radius label, and 479 hermes-agent PRs created 2026-10-02 to 10-05 shown twice (once in each half); hard-v1 4,000; devtools-v1 4,000; documents-v1 2,500; JGLUE 2,000; routing: agent factory 1,216, generic 1,000; decision-v7 replay 1,500: 23,427 records, none truncated | from stage 1, 1 epoch, lr 5e-5, states up to 5,120 tokens, 2,929 steps |
The mix is E2B v0.6's and E2B-it v0.6's, byte for byte (sha256 efff4b16), screened before training against every evaluation partition. Where it ran:
- Stage 1: steps 1–2,038 on one NVIDIA L4 (Hugging Face Jobs, bf16 autocast); steps 2,039–3,128 on an Apple M1 Max (MPS, fp32 forward), resumed from the saved step when the cloud job stopped.
- Stage 2: on the M1 Max (MPS, fp32 forward, bf16 frozen weights), finished 2026-10-17.
- LoRA and head master weights are fp32 on both machines, so only the forward precision changed between them.
provenance.json has the details.
Calibration: a single temperature, T = 1.55 (90% interval [1.48, 1.62]), fitted on pooled rows of the decision-v7 calibration split and the PR-labelling development set (3,178 questions). Calibration error (ECE) is 0.057 before and 0.015 after, out of fold. It is stored in d1a_config.json.
No routing temperature. A routing-specific temperature (T 1.20) was fitted and not shipped: out of fold it did not separate from the general one, and on held-out transfer it made calibration worse (ECE 0.018 → 0.031). Routing requests use T 1.55, the same outcome as E2B-it.
Results
Every D1A suite, both models as served (MLX 8-bit, unmerged, each at its own temperature), on a Mac mini M4. Δ is paired with 95% bootstrap intervals; bold where the interval excludes 0.
Table with columns: Suite, n, E2B-it v0.6, E4B-it v0.6, Δ [95% CI]| Suite | n | E2B-it v0.6 | E4B-it v0.6 | Δ [95% CI] |
|---|
| General decisions (decision-v7) | 1,264 | 84.8 | 85.7 | +0.9 [−0.7, +2.5] |
| Transfer (transfer-v4, sources never trained on) | 656 | 75.0 | 77.4 | +2.4 [−0.9, +5.9] |
| Hard decisions (hard-v1) | 1,083 | 69.2 | 75.7 | |
The live PR-labeler replay (98–193 questions) is flat on type and down on blast radius (64.3 → 58.2, not significant at this n). Against the older D1A-E4B v0.6.1 the comparison is indirect (both measured against E2B-it): E4B-it is ahead on decision-v7, transfer, hard and developer tools, and 1–2 points behind on long documents, JGLUE and dates.
Photos, voice and video
With the media extra, d1a.serving.serve answers questions about a photo, a voice clip or a video through Gemma 4 E4B-it's own vision and audio encoders, unchanged by the adapter (POST /v1/systemone/media). No media in training (zero-shot).
Measured on the playground's 14 photo, voice and video demo requests (28 answers): all answered. 7 answers differ from E2B-it v0.6; read by hand, 6 are better (it now calls the crushed-truck photo and video damaged, places the intact mailbox photo "in a mailbox", and no longer calls the English directions and Japanese not-at-home voice clips urgent) and 1 is worse (the intact front-door photo placed "on a sidewalk").
The questions it was trained for
PR labelling works best with these questions, worded exactly so, over a document of the form title / author / stats / body / files (one - status path +added/-deleted line per file):
type (choice): "Primary change type from files and body, not the title prefix." Options: type/bug, type/docs, type/feature, type/perf, type/refactor, type/security, type/test.
blast (choice): "How far a mistake in this PR spreads in production." Options: review:blast-contained (one module), -moderate (one subsystem), -broad (shared helper or config), -massive (auth, permissions, or all paths).
sev (choice): "How serious the problem this PR addresses is — not the risk of merging the diff as-is." Options P0 to P4.
The PR-labeler recipe has the exact wording, the document builder and the training and scoring scripts.
Run it
Needs d1a with jonpol01/d1a#261 (the <bos> above):
pip install "d1a[serve] @ git+https://github.com/jonpol01/d1a"python -m d1a.serving.serve --run JohnP1/d1a-e4b-it@v0.6 --port 8009
or in-process: from d1a import D1A; D1A.load("JohnP1/d1a-e4b-it@v0.6").decide(state, questions). Builds of the same model: MLX 8-bit JohnP1/d1a-e4b-it-mlx-q8 and MLX 4-bit JohnP1/d1a-e4b-it-mlx-q4, both with the photo, voice and video encoders and both merged (decisions only).
Enable chat
With its D1A adapter switched off, this model is Gemma 4 E4B-it, so one model in memory can answer D1A decisions (adapter on) and OpenAI-format chat (adapter off). This needs an unmerged build: the published MLX repos have the adapter merged into their weights and cannot chat. No chat build is published; build one from this adapter on Apple Silicon (downloads the E4B-it base, ~16 GB), with d1a from jonpol01/d1a#276 on:
git clone https://github.com/jonpol01/d1a && cd d1auv run --extra serve python scripts/export_mlx.py --run JohnP1/d1a-e4b-it@v0.6 --q-bits 8 --q-group-size 64 \ --unmerged --media --out runs/exports/d1a-e4b-it-chat
Serve it, with embeddings if your app needs them (mem0 does; the GGUF is nomic-ai/nomic-embed-text-v1.5-GGUF, file nomic-embed-text-v1.5.Q4_K_M.gguf, the one LM Studio serves):
uv sync --extra serve --extra embedD1A_EMBED_GGUF=<path to nomic-embed-text-v1.5.Q4_K_M.gguf> \ uv run --extra serve python -m d1a.serving.serve --run runs/exports/d1a-e4b-it-chat --port 8009
Then point any OpenAI client at http://localhost:8009/v1: POST /v1/chat/completions (streaming, tools, JSON mode by prompt), POST /v1/embeddings and GET /v1/models. Any model id is answered by this one model. Decisions stay on /v1/systemone with the adapter on. With d1a from jonpol01/d1a#291 on, a decision that arrives during a chat is answered between two of its tokens (0.24 s on this model during a 200-token answer, instead of waiting for the whole answer), and the chat's text is unchanged. Details: Chat with D1A.
Limits
- Slower than E2B-it: about 2.9× the model time per decision. Use E2B-it or E2B where latency matters more than accuracy.
- Calibration is mixed against E2B-it: ECE worse on 12 suites (largest +0.028 on Japanese PR test), better on 9.
- Unknowable facts (questions whose answer is not in the document): −2.9 against E2B-it.
- Gate tool calls through D1A's fail-safe thresholds (
d1a.agents.presets.advise: deny at p ≥ 0.5, allow only at p ≥ 0.8, otherwise ask), never through the raw top answer.
<bos>: requests encoded without the leading id 2 are out of distribution (see above).
- Media: zero-shot, measured on the playground's demo requests only.
- One temperature for every question, and no use-case temperatures (the routing one was measured and not shipped).
- The recent PR labels come from one large open-source project and a few days of it. If its conventions move again, so will these numbers.
- Questions are in English. The documents can be English or Japanese (trained on both).
Versions
Table with columns: Tag, What it is| Tag | What it is |
|---|
v0.6 | E2B v0.6's recipe on the Gemma 4 E4B-it trunk; decisions and chat in one model |
License and data
Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: github.com/jonpol01/d1a, built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.