birgermoell

sven-0.6b

README

License: apache-2.0

Try it + full stack

Table with columns: link, note
	link	note
🖥️ In-browser demo (no server, no keys)	birgermoell/sven-space	static Space; ONNX q4/fp16 in WebGPU→WASM; first load ~0.9 GB
🧮 Browser weights (ONNX)	birgermoell/sven-web	merged LoRA + pointer head; measured drift vs fp32: q4 ±0.024, fp16 ±0.008
📊 Training dataset + provenance	birgermoell/sven-data	5k translated records + 36 hand-written Swedish seeds + audit TSV
🔀 Local LLM-router recipe	repo: swedish-jev (git) — copy under `router/`	Sven first, Ollama when generation is needed; calibrated gates + JSONL decision log
🤝 Upstream (and credit)	jaredpalmer/kev	vendored Apache-2.0; deviations documented in VENDORED.md

Alla artefakter samlas i Hub-kollektionen "Sven — svensk System One".

Status: v0.1 (published), what changes next

Färdigt i v0.1: svensk/engelsk choice-score-noul med kalibrerade probabilities; tre publicerade utvärderingar nedan; ONNX-export med kvantparitet; demo + router.

Pågår / väntar (uppdateras här efterhand): v0.2 med 4B-backbone för score-frågorna, svenska router-proband-poster ((ticketlike/needs_llm-utforskningar är därför konservativa i demo-router:n), ev. QDQ-int4 för WebGPU-kärnfellet (orkar falla tillbaka till WASM, samma siffror).

Intended use

System One requests in Swedish or English: kundtjänst-routing, mejltriage, phishing/bedrägeri and prompt-injection-gardrails, incident-klassning, urgency/frustration-rubriker — the same mixed distribution as the upstream Kev recipe (banking77, MNLI, SST-5, BoolQ, …), machine-translated to Swedish plus hand-written Swedish seed records. Serve with the vendored Kev server: python -m kev.serve --run <this repo> --port 8009 (TypeSafe-compatible /v1/systemone).

Results

Evaluated fp32 (MPS), frozen upstream dev partitions; Swedish split held out per record.

Table with columns: eval, accuracy, Brier, ECE, selective acc @50% cov, n (rejected)
eval	accuracy	Brier	ECE	selective acc @50% cov	n (rejected)
Swedish held-out	0.745	0.331	0.079	0.947	400 (8)
└ choice	0.751	—	—	—	—
└ noul (yes/no)	0.863	0.225	0.075	0.986	—
└ score (0–4 rubrics)	0.520	0.556	0.099	—	—
English in-distribution (decision-v7 dev)	0.794	0.299	0.090	0.959	1,204
English out-of-domain (transfer-v4 dev)	0.623	0.536	0.146	0.726	764

Upstream reference, same suites (Kev-0.6B, English): 0.801 in-distribution / 0.620 out-of-domain acc; Brier 0.266 / 0.483 (locked test). Sven retires < 1 point of upstream English accuracy with the Swedish delta applied — the fine-tune does not wash out the base recipe.

Rejected records (state/branch over Kev's serving context, e.g. 77-option routing with long Swedish criteria) count as wrong; 8/400 = 2% on the Swedish split.

Training

Base: Qwen/Qwen3-0.6B-Base @ da87bfb6… (Apache-2.0), warm-started from jaredpalmer/kev-0.6b (392 LoRA tensors + pointer head), delta LR 2e-5, 2 epochs, batch 1 × grad-accum 8, fp32 master weights, cross-entropy on the option distribution (upstream protocol; no Jev outputs used).
Data: 5,000 EN→SV machine-translated records (stratified sample of upstream decision-v7 train; MT: Helsinki-NLP/opus-mt-en-sv, unique-string cache; option keys kept English with Swedish criteria text) + 36 hand-written Swedish seed records (Swedish option keys; routing, phishing, incident, GDPR and prompt-injection domains) + 200 English replay records from the upstream suite to prevent recipe drift.
Provenance: sven-data dataset (train/held-out split sha256 of suite, per-source counts, MT audit sample of 30 EN→SV pairs).

Data & licenses

Everything open: Apache-2.0 (code + weights) per upstream Kev; Qwen3-0.6B-Base Apache-2.0; OPUS-MT translator open. Translated training data is redistributed for research with per-source documentation in DATA_LICENSES.md (Banking77 CC-BY-4.0, BoolQ/MNLI/DBpedia CC-BY-SA, etc.); dataset release ships provenance so stricter per-source terms can be traced. Hand-written Swedish seeds and this card: CC-BY-4.0 / Apache-2.0 respectively.

Known limits

Score questions are the weak spot (0.52 on translated 0–4 rubrics): the length of Swedish criteria plus 0.6B capacity — not data volume — is the binding constraint (upstream notes the same physics). The documented fix is the 4B backbone (planned v0.2).
MT-translated labels inherit MT noise; review audit_sv.tsv (sven-data) for a human-checked sample. One long-review record showed visible MT truncation.
Most translated records keep English option keys (fakturering-style keys only in seeds), so Swedish option-key handling is narrower than the state/question coverage.
Probabilities are advisory out-of-domain (ECE 0.15 transfer) — gate automation on confidence (at ≥0.9 confidence the Swedish split lands ~0.90 accuracy on the covered share).
Same architecture caveats as upstream: option-order sensitivity remains; questions cannot read each other by design; latency unchanged from Kev-0.6B (0.12 s / five questions on M-series).

Use

python

from typesafe import TypeSafeClientclient = TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8009", model="sven-latest")

bash

KEV_DTYPE=bf16 python -m kev.serve --run <HF repo or local run dir> --port 8009

License

Apache-2.0 for the adapter and head (upstream Kev license). Base model Apache-2.0 (Qwen3). Built with the vendored Kev codebase (see VENDORED.md).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

birgermoell

Model Tree

Base

Qwen/Qwen3-0.6B-Base

Adapter

this model

Input Modalities

Text

Output Modalities