The cut
Leaders ship v1 and changelog the rest. We name the retraining of silence as its own model. Failure is an artifact, not a footnote.
A public retrain whose only job is to improve one metric: honest abstain under adversarial handles.
Silhouette → leave → SZL
Table with columns: Leader, Take, then tweak| Leader | Take, then tweak |
|---|
| Anthropic | Red-team → constitution update. We red-team → adapter. |
| NVIDIA | Recipe re-run with a new seed and a bounded delta. |
| Unsloth | Same FastLanguageModel loop, new curriculum, new evidence record. |
This card makes the combination inspectable: a separate abstain-retraining SKU, bounded small-sample results, and an exact-source documentation publisher that cannot modify weights or promotion state.
Intended use
Continue the abstain-retrain loop. Keep publication and autonomy gates closed until independently verifiable evaluation evidence satisfies the governed release criteria.
Limitations
- research-only
- No signed R2 eval receipt in this atelier.
- Publishing this card is a documentation update, not model promotion or autonomy approval.
Canonical GitHub: szl-holdings/szl-forge/khipu_r2
Specification
Evaluation (MEASURED this job)
Method: in-process Unsloth generate, scoring ported from eval_khipu.py, temperature 0, held-out never in gradients. Host job worker. Date 2026-08-28 17:20 UTC. File: eval_measured.json.
Table with columns: split, k/n, what-NOT| split | k/n | what-NOT |
|---|
| plan-valid | 11 / 11 | not a public leaderboard |
grounding (eval.jsonl navigate) | 5 / 5 | n=5 |
abstain (adversarial.jsonl) | 3 / 6 | not 5/5, not 6/6 |
| hallucinated citations | 0 | this job only |
Prior published original SZL-Khipu-1.5B MEASURED abstain was 2/6. This run is 3/6. Small n. Do not derive a world-rank score from k/n on n=11.
Training (MEASURED / REPORTED)
- Unsloth QLoRA, seed 11, lr 2e-4, LoRA r=32 α=64, 45 epochs
- Train: 15 navigate + 8 abstain rows × oversample 4 (in-memory 32)
- Held-out: 5 + 6,
held_out_in_gradients: false
training_loss MEASURED 0.017188… is a train metric, not an eval
- adapter sha256
e44d53f29f2d443598e06d6c0441557fd3a5010888c7aa97b56ec3c0e050d349
What this is NOT
- Not a replacement for
SZL-Khipu-1.5B
- Not Chaski (Qwen3.5 lock)
- Not an autonomous agent
- Not a GGUF. Mini GGUFs exist on
A11OY-MINI; Mini evals none-this-run; Mini does not inherit this 3/6
Load
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_id = "Qwen/Qwen2.5-1.5B-Instruct"
tok = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base, "SZLHOLDINGS/KHIPU-R2")
Owner: Stephen Lutar / SZL Holdings.