Use with GiTrust
# ~/.gitrust.tomlkev_run = "AniketDalvi385/gitrust-kev-0.8b@v2"
GiTrust starts the Kev server itself; the weights are downloaded to the Hugging Face cache on first
use. Directly with a Kev checkout:
uv run --extra serve python -m kev.serve --run AniketDalvi385/gitrust-kev-0.8b@v2 --port 8009
The options must be GiTrust's, verbatim and in order. The model was trained on the exact option
wording and order in kev_ops.json (also in this repo). Rewording or reordering them
puts it off its training distribution; it is not a general Git classifier for arbitrary option lists.
Training
Table | |
|---|
| Initialised from | jaredpalmer/kev-0.8b@v1.0 (LoRA r = 16 and the pointer head, warm-started) |
| Data | 1,481 synthetic requests over 36 options, written for GiTrust (make_kev_train.py --round 2): paraphrases, typos, casual wording, out-of-scope requests, near-miss pairs |
| Recipe | kev.train, 2 epochs, lr 2e-5 (adapter and head), permutation-consistency loss (perm_kl 1.0 on 30% of rows), bf16, one Colab T4 |
| Calibration | temperature 4.0, fitted on the 259-row dev set and stored in head.pt |
| Hold-out | every evaluation phrasing (and anything within Jaccard 0.6 of one) removed from the training data; the evaluation's slot values (topics, branch names, counts, messages) banned |
Evaluation
All numbers are on author-labelled requests that were never trained on. They are small sets; read
them as "this works for GiTrust's requests", not as a benchmark.
Table with columns: stock Kev-0.8B, this model | stock Kev-0.8B | this model |
|---|
| Dev set (259 rows, held out from training) | 186 (71.8%) | 241 (93.1%) |
| 44 held-out requests, 36 options, clean / dirty repo | — | 41 / 41 |
| Out-of-scope and safety items | — | 4/4 · 5/5 |
Inside GiTrust (thresholds, menus and gates included), on 68 requests written by the owner: 48 acted
on correctly, 8–9 answered with a menu, 1 wrong read-only action, 0 wrong actions that change
anything, in both a clean and a dirty repository.
Known weak spots: "psuh" (typo of push) is read as not-Git; "roll back that last commit i just made"
leans to revert instead of undo (below GiTrust's destructive bar, so it becomes a menu); terse
requests such as "whats goin on" get a menu instead of status. The full write-up, per-case results
and the training notebook are in GiTrust's
docs/kev_migration_research.md
(§12) and docs/classifier_eval/.
Limitations
- English only; tuned for one developer's phrasing plus synthetic variants.
- Not safe to act on alone. Its value in GiTrust comes from the deterministic layers around it: exact
phrases first, confidence bars, a re-ask with the options reversed for destructive routes, and
per-operation confirmations.
License
Apache-2.0, like Kev and the Qwen3.5 base. Kev's own training data is described in its
model card.