SlayerLab

bielik-4.5b-v3-matura-history-sft2

README

License: apache-2.0

Files

Table with columns: File, Bytes, Notes, sha256
File	Bytes	Notes	sha256
`model-00001-of-00002.safetensors`	4,975,516,024	bf16, Transformers format (+ `config.json`, tokenizer, `chat_template.jinja`, index)	`fc70b9a7e415164087bfd8f2f25189de35cfdfa41574d2d2a7e48cfc88704edb`
`model-00002-of-00002.safetensors`	4,539,113,296		`52a8fd2d1ff17f2e427aa8e448fbeab84ec9bbd9955b74d48dbb01dabbe428a9`
`sft45-bf16.gguf`	9,519,301,024	`convert_hf_to_gguf.py`; the source of every quant below	`5f02a3207d9e29d80ac5596100352afaef9419b6cb88268f259c6140e07bb08a`
`sft45-IQ4_XS.gguf`	2,558,825,216	Best scores in this repo	`d5df84995ddfd912129e85b0b1eb14b6848ab3b9db959fca43a5c64147e357bc`
`sft45-Q3_K_M.gguf`	2,303,443,712	Not evaluated	`fe8ff71654f302eea1a40e0e135519f0d7c9142e255e2121082821cae21aeb8d`
`sft45-IQ3_XXS.gguf`	1,851,896,576	Registered candidate	`9e3254c4e6cf95d4f07a5d3f6ed239dc9c4182ae7a4e49858edc18c7ac19d1f4`
`sft45-IQ2_M.gguf`	1,615,532,800	Fails the past-exam track	`b4cd6325e2f1e76ba473097ca50806c8329f4aa60710e76d6b6fd429181c13e6`
`sft45.imatrix.gguf`	5,655,680	Polish importance matrix: SFT conversations + plwiki passages (`train/make_calib.py`, `calib-v1.txt`)	`73fb9b8faf420a6fee7e791482ac040ce4d26b79fbc7a48df18ac8f9e6815f49`
`training/`		`train_args.json`, `metrics.json`, `log.jsonl`, `data_stats.json`, `gpu.txt`, `out-pip-freeze.txt`
`SHA256SUMS`		sha256 of every file in this repo (`sha256sum -c SHA256SUMS`)

Quants were made with llama.cpp build 11146 (commit 7fe450e19) via train/quantize.sh, with the same Polish calibration text as the 1.5B (calib-v1.txt).

How to run

The configuration that was scored (llama.cpp llama-server, OpenAI-compatible API on port 8891):

hf download SlayerLab/bielik-4.5b-v3-matura-history-sft2 sft45-IQ3_XXS.gguf --local-dir .llama-server -m sft45-IQ3_XXS.gguf --port 8891 --ctx-size 8192 -ngl 99 --flash-attn on --jinja \  --chat-template-kwargs '{"enable_thinking":false}'

llama-server -hf SlayerLab/bielik-4.5b-v3-matura-history-sft2:IQ3_XXS and ollama run hf.co/SlayerLab/bielik-4.5b-v3-matura-history-sft2:IQ3_XXS also work.

The reported harness scores used matura-harness in front of that server with the flags in maly-ale-wariat/eval/harness_args_v5.txt (evaluator-style prompts for source-based tasks, plwiki BM25 retrieval of 2 × 700 characters only for tasks of 600 characters or less and for essays, strict closed-answer normalization, repetition trimming, DRY sampling and a 380-word floor for essays). The 1.7 GB retrieval index is not part of the model; the steps to rebuild it are in the repo README.

Transformers (bf16 safetensors):

python

from transformers import AutoModelForCausalLM, AutoTokenizer repo = "SlayerLab/bielik-4.5b-v3-matura-history-sft2"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="auto") messages = [    {"role": "system", "content": "Jesteś zdającym maturę rozszerzoną z historii. Odpowiadasz po polsku, zwięźle i konkretnie. "                                  "Najważniejsze są źródła podane w zadaniu; materiały pomocnicze mogą być nietrafne. "                                  "Wykonaj dokładnie polecenie i podaj wymaganą liczbę elementów."},    {"role": "user", "content": "Podaj nazwę bitwy z 1410 r., w której wojska polsko-litewskie pokonały zakon krzyżacki, "                                "i wymień jednego z dowódców strony polsko-litewskiej."},]ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=200, do_sample=False)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

The system prompt above is the one used in training. Chat template: Bielik ChatML (<s><|im_start|>role\n…<|im_end|>), no thinking mode. Context: 32768 tokens (the scored runs used 8192).

Evaluation

Team evaluator history-matura-eval: two tracks of 55 points each, 34 items adapted from the CKE May 2023 extended-level history paper ("past exam") and 34 Wikipedia-based items, graded by a deterministic closed-answer scorer plus an LLM judge (google/gemini-3.1-flash-lite). Pass = at least 20/55 (35%) on each track. Two protocols:

raw: the evaluator's bare-model runner (its system prompt, the task text only, the answer submitted verbatim);
harness v5: matura-harness with eval/harness_args_v5.txt, as described above.

A third, truly unseen paper, CKE May 2026 (39 items, 60 points, pass = 21), was converted to the evaluator's item format and scored with the same scorer and judge. Scores below are strict re-scores (eval/strict_rescore.py); the value the evaluator reported is in brackets where it differs.

Table with columns: GGUF, Size, Protocol, Past exam (/55), Wikipedia (/55), CKE May 2026 (/60)
GGUF	Size	Protocol	Past exam (/55)	Wikipedia (/55)	CKE May 2026 (/60)
`sft45-IQ3_XXS`	1.85 GB	harness v5	22 ✅	34 ✅	23 ✅
`sft45-IQ3_XXS`	1.85 GB	raw	incomplete¹	34 ✅	22 ✅
`sft45-IQ4_XS`	2.56 GB	harness v5	25 (27) ✅	37 ✅	23 ✅
`sft45-IQ4_XS`	2.56 GB	raw	–	–	24 ✅
`sft45-IQ2_M`	1.62 GB	harness v5	14 ✗	24 ✅	–
`sft45-IQ2_M`	1.62 GB	raw	12 ✗	21 ✅	–
base `Bielik-4.5B-v3.0-Instruct` Q8_0	5.06 GB	harness v5	41 ✅	38 ✅	38 ✅
base `Bielik-4.5B-v3.0-Instruct` Q8_0	5.06 GB	raw	28 ✅	40 ✅	–
base `Bielik-4.5B-v3-Instruct` i1-IQ3_XXS²	1.85 GB	harness v5	18 ✗	35 ✅	–

¹ The evaluator reports no total when the judge errs or grades an item "review". ² mradermacher's imatrix quant of the ungated mirror, used as the size-matched untouched baseline.

What this shows: the untouched 4.5B has real margin at Q8 (5 GB) but loses most of it below 4 bits (18/55 at IQ3_XXS). The SFT recovers the past-exam track at that size (22/55) and holds 22–23/60 on the unseen 2026 paper, where every 1.5B variant sits at 19–21. The price is a file twice the size of the 1.5B candidate.

Caveats

A single evaluator run varies by about ±3 points; the judge is an LLM. The 22/55 past-exam pass has a 2-point margin.
The bare-runner past-exam score for IQ3_XXS is unavailable (judge error voided the track), so "passes under both protocols" rests on the Wikipedia track and the 2026 paper for the bare protocol.
IQ2_M fails; Q3_K_M was not evaluated.
Known failure modes: true/false statements near chance level, open answers naming the wrong person/place/date, essays with factual errors.

Training

Table

Base	`speakleash/Bielik-4.5B-v3.0-Instruct` (Apache-2.0), loaded from the ungated mirror `adamo1139/Bielik-4.5B-v3-Instruct-ungated`. Both safetensors shards match the LFS sha256 of the gated original (`bae33275…`, `e66d692a…`), so the base is verified identical.
Method	Full-parameter SFT (`train/sft.py`, plain `transformers` 4.57.1), loss only on assistant tokens. Same script and data as the 1.5B.
Data	4,144 chat rows (`sft/v2`, prompt version `compact-v2`) + 116 validation rows: 2,714 synthetic syllabus tasks, 300 model essays (×2), 800 supplementary items, and 82 items converted from the CKE May 2024/2025 papers (×2; not redistributed, CKE copyright). 66% compact prompt with 0–2 retrieved plwiki passages, 20% evaluator-style bare prompt. 2.80M training tokens, 576k of them supervised.
Hyper-parameters	2 epochs, lr 1e-5 cosine to 0.1×, warmup 3%, weight decay 0, batch 4 × grad-accum 8 (effective 32), max length 4096, bf16, gradient checkpointing, seed 42.
Compute	1 × NVIDIA A100 80GB PCIe (RunPod), 260 steps, 1,798 s (30 min).
Loss	train 1.147, eval 1.155 (final; curves in `training/log.jsonl`).

Data provenance and licensing

Weights: Apache-2.0, same as the base model.
The synthetic tasks and essays were written by Claude (Anthropic) via Claude Code workflows following data/gen/STYLE.md, fact-checked by a second agent pass and validated by data/gen/validate_items.py. They are CC BY-SA 4.0-compatible.
Retrieved evidence comes from Polish Wikipedia (CirrusSearch dump 2026-09-20, CC BY-SA 4.0); every record keeps its source URL.
The CKE 2024/2025 items are not redistributed (CKE copyright). The evaluator's questions and keys were never used for training; build_sft.py drops any record sharing word 8-grams with them, plus essays on the held-out themes.

Intended use and limitations

A hackathon research artifact: a small Polish model that answers exam-style history questions in the CKE matura format. It hallucinates names, dates and places, answers true/false items near chance, and was tuned for a specific evaluator. Do not rely on it for factual information or grading.

Acknowledgements

SpeakLeash for Bielik-4.5B-v3.0-Instruct; llama.cpp for quantization and inference; Polish Wikipedia authors (CC BY-SA 4.0).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

SlayerLab

Model Tree

Base

speakleash/Bielik-4.5B-v3.0-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities