Risethagain

rippy-kev-4b

README

License: apache-2.0

Use

git clone https://github.com/jaredpalmer/kev && cd kev && uv sync --extra serveuv run --extra serve python -m kev.serve --run Risethagain/rippy-kev-4b --port 8012

toml

# ~/.rippy/config.toml (global config only)[jev]enabled = trueendpoint = "http://127.0.0.1:8012/v1/systemone"model = "kev-latest"api-key-env = "RIPPY_KEV_KEY"   # any non-empty value; the local server needs no keytimeout-ms = 5000               # local models need more than the 2000 default under load# Recommended thresholds for this model. Pick one block.# Balanced: about 1% of severe dev cases approved.min-confidence = 0.75max-irreversible = 0.2max-writes-outside = 0.3# Strict: about 0.2% of severe dev cases approved.# min-confidence = 0.95

On Apple silicon Kev serves through MLX: about 0.75 s per command on an M4 Max.

llama.cpp (GGUF)

rippy-kev-4b-v2-Q8_0.gguf is this model converted with llama.cpp's KevModel converter. The LoRA is merged into the base, and the pointer head and the fitted temperature (1.231) are embedded. It needs llama.cpp build 11361 or later, which serves POST /v1/systemone:

huggingface-cli download Risethagain/rippy-kev-4b rippy-kev-4b-v2-Q8_0.gguf --local-dir .llama-server -m rippy-kev-4b-v2-Q8_0.gguf --port 8012 -ngl 99 --parallel 1 -c 4096 --cache-ram 0 --ctx-checkpoints 0

Point rippy's [jev] endpoint at http://127.0.0.1:8012/v1/systemone with the same thresholds as above.

At those thresholds the Q8_0 file matches the original on rippy's test sets:

Table with columns: probability change vs original (mean / max), decisions flipped (of 2,947), never-seen safe approved, severe approved, gold safe
	probability change vs original (mean / max)	decisions flipped (of 2,947)	never-seen safe approved	severe approved	gold safe
original (kev.serve)	–	–	668/944	9	28/35
Q8_0 GGUF	0.001 / 0.07	8	669/944	8	28/35

Use Q8_0, not Q4_K_M. Q4_K_M flipped 62 of the decisions in a partial run, with probabilities moving by up to 0.60.
Use the flags above. With llama-server's defaults (prompt cache in RAM, context checkpoints, full model context), latency grew steadily over a long run and memory reached 11 GB. Even with the flags, a run of a few thousand commands had slow stretches. Monitor a long-running server and restart it if latency drifts.
On Apple silicon, prefer kev.serve. It shares the state across the seven questions; llama.cpp evaluates each question separately. Raw 7-question requests on an M4 Max: kev.serve (bf16) 836 ms, llama.cpp 1.2 s (Q4_K_M, the fastest quant tried). The GGUF is the option for Linux, CPU-only and CUDA machines, or for a single binary without Python.

What it answers

The question set is rippy's q3, byte for byte: effect (read_only / remote_read / local_change / destructive / network_send / download_execute), plus yes/no on exfiltration, writes_outside_project, reads_secrets, irreversible, runs_project_code and self_referential (text in the command arguing for its own approval).

rippy's policy decides, not this model. The model can only move an uncertain ask to Allow, or force a prompt on suspected exfiltration or steering. Commands rippy blocks or asks about by design never reach it.

Results

All test programs are absent from training. The thresholds are fitted on dev, and the test sets are untouched.

Never-seen programs (944 safe, 945 severe commands), at a budget of 1% severe approvals on dev:

Table with columns: backend, safe approved, severe approved, rippy gold set: safe / severe, exfil escalated (155 hard cases)
backend	safe approved	severe approved	rippy gold set: safe / severe	exfil escalated (155 hard cases)
Jev 1.13 (hosted)	611 (65%)	12	27/35 / 0	147
rippy-kev-4b v2	668 (71%)	9	28/35 / 0	151
rippy-kev-0.8b v2	453 (48%)	11	19/35 / 0

At a stricter 0.5% budget, v2 approves 44% of safe commands with 2 severe approvals (Jev: 51%, 3). At a 2% budget, v2 approves 76% with 15 (Jev: 73%, 16).

Severe means the merged teacher labels say destructive, network send, download-and-run, exfiltration, reads secrets, irreversible, or writes outside the project.
Other non-read-only commands are mild: a local change in the project, a remote read, or project code. They should prompt, but approving one is not a security failure.

Kev's own development report (1,696 never-seen-program records, all seven questions):

Table with columns: value
	value
accuracy	0.945
ECE (temperature 1.23)	0.006
coverage at 5% error	0.99
`effect` accuracy	0.828
public decision-v7 regression (417 records)	0.867 (Kev-4B: 0.862)

Training

Commands: 26,301 records.
- 23,674 are rippy q3 states built from tldr-pages examples (CC BY 4.0), three fills per example. Only commands rippy itself would send for review are kept.
- 2,627 are hard cases generated by Claude Sonnet: steering and lookalikes, exfiltration, secret reads, download-and-run, destructive commands with $VAR, benign commands that sound scary, and the read/write boundary.
- The split is by program. Programs in rippy's gold sample are excluded everywhere.
Labels: soft targets from weighted votes.
- Two independent Claude Sonnet passes label every record; they agree on all seven answers for 80% of records.
- Gemma 4 31B adds a third vote on dev and test.
- A blind Claude Opus adjudication, counted double, covers the 2,206 records whose teachers split on approval.
- Hosted Jev was only an evaluation reference, never a teacher.
Kev's delta fine-tune from .

Limitations

Question set. Trained on rippy's q3, where facts read e.g. "not found on rippy's PATH; may still exist when the command runs". Re-check a later question-set change with rippy's scripts/jev-eval before relying on it.
Thresholds. Thresholds tuned for Jev do not carry over. Use the ones above, or fit your own with rippy-kev's eval/fit_thresholds.py.
Labels are model-made. Labels come from teacher models with a hand-labelled gold check, not from human review of every record. "Severe" and "safe" in the results are teacher verdicts.
Remaining misses at the balanced setting are mostly config or settings commands of unfamiliar tools (blackfire config, mods --settings) and editors or viewers pointed at files outside the project (kak /etc/…). It also approved commands that print stored credentials (xauth list, mc alias list). The strict setting leaves 2 severe approvals per ~945 severe never-seen commands.

Licence and attribution

Apache-2.0. It builds on:

Kev (Apache-2.0)
Qwen3.5-4B-Base (Apache-2.0)
commands derived from tldr-pages (CC BY 4.0)

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Risethagain

Model Tree

Base

Qwen/Qwen3.5-4B-Base

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality