Results (held-out, vs the base model)
Table with columns: Test, What it measures, Base, bev| Test | What it measures | Base | bev |
|---|
| Risk (16,902 questions) | credentials and destructive content in ops text | 93.5% | 97.0% |
| Evasion holdout (90) | evasion attempts never seen in training | 65.6% | 93.3% |
| Red team (34) | adversarial commands | 53% | 74% |
| Jev bench (1,200) | general Jev questions, outside our domain | 80.6% | 82.8% |
The test splits come from the same corpora as training — they measure fit
to this kind of traffic, not yours. The evasion and red-team sets are
small (90 and 34 cases). Adversarial commands sit at 74%, which is why the
gate is one layer among several.
The model keeps CLEF-Flash's joint schema architecture: a single
forward pass answers a map of typed questions (noul/choice/score) over a
shared state. Loading needs the serving stack
(joint_schema_model.py), not AutoModelForCausalLM:
- Load the base model with the CLEF-Flash serving code
- Attach this adapter with
peft (adapter_config.json targets 7
backbone modules)
- Replace the head weights with
joint_head.safetensors
The full serving stack is embedded in
Berget AI's System One API; the plugin talks
to it over the Jev POST /v1/systemone contract.
Training
- Data: 221,759 judged decisions from real operations traffic (Swedish +
English), labels validated by a stronger model (Kimi-K3) and human
review. No raw customer data ships with these weights.
- LoRA: r=16, alpha=32, 7 target modules on the language backbone.
- The joint head was fine-tuned alongside the adapter on the same data.
License
Apache-2.0, matching the base model.