Remek

basal-1.0-1.5B

README

License: apache-2.0

Quick start

1. Install into a fresh uv environment (no git needed):

bash

uv venv --python 3.12 ~/basal-env && source ~/basal-env/bin/activateuv pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v1.0.1.tar.gz"

Install torch first, from the CUDA 12.8 index: the newest torch on PyPI may need a newer GPU driver, and the torchvision preinstalled on cloud GPU images breaks transformers. DGX Spark and B300: see installation.

2. Start the server and wait until it prints basal: model ... ready:

bash

basal-serve --model Remek/basal-1.0-1.5B --mode fast --port 8000

The first start in mode fast compiles the model for a few minutes; --mode fast-nocompile starts in seconds.

3. Ask a question (in a second terminal):

bash

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej.",  "questions": {"dept": {"type": "choice", "instructions": "Do którego działu skierować zgłoszenie?",    "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"}}}}'

The response has a calibrated probability for every option, the chosen key and the confidence:

json

{"model": "basal-1.0-1.5B", "answers": {"dept": {"type": "choice", "choice": "online",   "probabilities": {"cards": …, "online": …, "loans": …}, "confidence": …}}, "usage": {"input_tokens": …, "output_tokens": 0, "questions": 1, "latency_ms": …}}

From Python (the basal package includes a client):

python

from basal.client import Basalb = Basal("http://127.0.0.1:8000")state = "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej."a = b.choice(state, "Do którego działu skierować zgłoszenie?",             {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"})print(a["choice"], a["confidence"])                                      # chosen key and its probabilityprint(b.yes_no(state, "Czy klient zgłasza problem techniczny?")["noul"])  # P(yes)print(b.score(state, "Jak pilne jest zgłoszenie?", ["niska", "średnia", "wysoka"])["score"])  # expected level 0-2

Many items at once: basal-run --input items.jsonl --output answers.jsonl (JSONL format). Several questions per request, option keys, early exit and the full response format: API.

Quality

Table with columns: system, params, PL decisions, PL general, EN decisions, Public bench.
system	params	PL decisions	PL general	EN decisions	Public bench.
basal-1.0-4.5B	4.5B	0.884	0.737	0.741	0.740
basal-1.0-1.5B	1.5B	0.849	0.656	0.734	0.675
Jev 1.13.0 (commercial API)	–

PL decisions: 7,081 held-out Polish decisions from unseen templates, statutes and domains; PL general: Polish knowledge, exams and reading comprehension; EN decisions: 1,479 held-out English decisions; Public bench.: the 231-item public English decision benchmark (official harness). All systems served on one H100 with their own servers; both option orders averaged. Leaderboard: jev-pl-benchmark. The basal public-benchmark scores are measured with engine v1.0.1, which shows option keys next to their descriptions by default (the benchmark's options have meaningful keys); with v1.0 they were 0.706 (4.5B) and 0.662 (1.5B).

Speed (one decision = both option orders, batch size 1)

Table with columns: GPU, fast (bf16), fp8, basal-1.0-4.5B bf16
GPU	`fast` (bf16)	`fp8`	basal-1.0-4.5B bf16
B300 SXM6	4.7 ms, 250 dec/s	5.2 ms, 224 dec/s	8.8 ms
H100 80GB	6.2 ms, 157 dec/s (HTTP: 7.7 ms, 147 dec/s)	6.3 ms, 177 dec/s	12.5 ms
RTX PRO 6000 Blackwell	8.8 ms, 101 dec/s	7.6 ms, 128 dec/s	19.1 ms
RTX 5090

1.9–2.7× faster than the 4.5B model. Agreement with the fp32 reference: 0.99–1.00 in bf16, 0.96–0.97 in FP8. FP8 and NVFP4 checkpoints for vLLM: Remek/basal-1.0-1.5B-FP8, Remek/basal-1.0-1.5B-NVFP4 (NVFP4 costs quality, see its card).

How it works

The model receives a fixed chat prompt with the state, the question and lettered options; the assistant turn is prefilled with {"answer": " and the decision is the softmax over the next-token logits of the option letters only (one forward pass, no text generation). The server asks every question with the options in original and reversed order and averages the two distributions (reduces sensitivity to option order), then applies the calibrated temperature of the question type from CALIBRATION.json, fitted on exactly this averaged prediction (calibration v1.0.1).

Use the confidence. CALIBRATION.json also stores confidence thresholds chosen on the calibration split, before testing, for a target error of 1% or 5% among accepted decisions; applied once to the test split they accept 49.5% / 68.6% of test decisions at 1.4% / 5.5% observed error. Accept decisions above the threshold automatically and route the rest to a person; with your own data, refit the thresholds on a labelled sample. These numbers were measured on descriptions-only prompts ("option_keys": "hide"); in the default mode, which also shows option keys, they are not validated — refit the thresholds on your own labelled requests.

Training data

Polish and English decision data whose labels are computed by code (deadlines, amounts, rule families with twin pairs that differ in one fact), grounded in statutes (verbatim evidence quotes checked by independent verifiers), or agreed by independent verifier models; plus English decisions from a public dataset (about 21% of the training items). Generated data are split by template, statute and domain, so test items come from templates, statutes and domains never seen in training; the English items follow the source corpus's own train/test split. Part of the data was generated or verified with commercial models.

Limitations

Evaluated on held-out items from the same generation pipelines as training plus public benchmarks; validate on your own documents before relying on it.
Polish world knowledge of a small model is limited: provide the relevant facts in the state.
Legal rules change; the model does not know rules introduced after its training.
A generator error in the training data taught the model the wrong notice period (art. 36 § 1 KP) when three years of employment are completed during a one-month notice: it answers one month instead of three. Evaluation labels are corrected; the model will be retrained in the next release.
Averaging the original and reversed option order reduces, but does not remove, sensitivity to option order for three or more options.
About 21% of the training items (13,500 English items) come from an aggregated public corpus whose upstream sources could not be traced item by item; see the technical report.
The test split was consulted during development; its results come from an adaptive process on held-out templates, not from a single untouched final evaluation.
Decisions with serious consequences for people should be reviewed by a person.

Citation

bibtex

@techreport{kinas2026basal,  title       = {basal-1.0: Reliable, Highly Optimized Typed Decisions for Polish},  author      = {Kinas, Remigiusz},  institution = {ai5},  year        = {2026},  type        = {Technical report},  doi         = {10.5281/zenodo.23022986},  url         = {https://doi.org/10.5281/zenodo.23022986}}

License and attribution

Apache-2.0. Fine-tuned from speakleash/Bielik-1.5B-v3.0-Instruct (Apache-2.0).

Training data: English decision items were converted from avbiswas/bev-decision-150K, which aggregates questions derived from many upstream sources; its maintainers ask users to check and attribute those sources, and row-level source identifiers are not available (see the technical report).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Remek

Model Tree

Base

speakleash/Bielik-1.5B-v3.0-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Quick start

1. Install into a fresh uv environment (no git needed):

bash

uv venv --python 3.12 ~/basal-env && source ~/basal-env/bin/activateuv pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v1.0.1.tar.gz"

2. Start the server and wait until it prints basal: model ... ready:

bash

basal-serve --model Remek/basal-1.0-1.5B --mode fast --port 8000

The first start in mode fast compiles the model for a few minutes; --mode fast-nocompile starts in seconds.

3. Ask a question (in a second terminal):

bash

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej.",  "questions": {"dept": {"type": "choice", "instructions": "Do którego działu skierować zgłoszenie?",    "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"}}}}'

The response has a calibrated probability for every option, the chosen key and the confidence:

json

{"model": "basal-1.0-1.5B", "answers": {"dept": {"type": "choice", "choice": "online",   "probabilities": {"cards": …, "online": …, "loans": …}, "confidence": …}}, "usage": {"input_tokens": …, "output_tokens": 0, "questions": 1, "latency_ms": …}}

{"model": "basal-1.0-1.5B", "answers": {"dept": {"type": "choice", "choice": "online",   "probabilities": {"cards": …, "online": …, "loans": …}, "confidence": …}}, "usage": {"input_tokens": …, "output_tokens": 0, "questions": 1, "latency_ms": …}}

From Python (the basal package includes a client):

python

from basal.client import Basalb = Basal("http://127.0.0.1:8000")state = "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej."a = b.choice(state, "Do którego działu skierować zgłoszenie?",             {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"})print(a["choice"], a["confidence"])                                      # chosen key and its probabilityprint(b.yes_no(state, "Czy klient zgłasza problem techniczny?")["noul"])  # P(yes)print(b.score(state, "Jak pilne jest zgłoszenie?", ["niska", "średnia", "wysoka"])["score"])  # expected level 0-2

Many items at once: basal-run --input items.jsonl --output answers.jsonl (JSONL format). Several questions per request, option keys, early exit and the full response format: API.

Quality

Table with columns: system, params, PL decisions, PL general, EN decisions, Public bench.
system	params	PL decisions	PL general	EN decisions	Public bench.
basal-1.0-4.5B	4.5B	0.884	0.737	0.741	0.740
basal-1.0-1.5B	1.5B	0.849	0.656	0.734	0.675
Jev 1.13.0 (commercial API)	–

Speed (one decision = both option orders, batch size 1)

Table with columns: GPU, fast (bf16), fp8, basal-1.0-4.5B bf16
GPU	`fast` (bf16)	`fp8`	basal-1.0-4.5B bf16
B300 SXM6	4.7 ms, 250 dec/s	5.2 ms, 224 dec/s	8.8 ms
H100 80GB	6.2 ms, 157 dec/s (HTTP: 7.7 ms, 147 dec/s)	6.3 ms, 177 dec/s	12.5 ms
RTX PRO 6000 Blackwell	8.8 ms, 101 dec/s	7.6 ms, 128 dec/s	19.1 ms
RTX 5090

How it works

Training data

Limitations

Evaluated on held-out items from the same generation pipelines as training plus public benchmarks; validate on your own documents before relying on it.
Polish world knowledge of a small model is limited: provide the relevant facts in the state.
Legal rules change; the model does not know rules introduced after its training.
A generator error in the training data taught the model the wrong notice period (art. 36 § 1 KP) when three years of employment are completed during a one-month notice: it answers one month instead of three. Evaluation labels are corrected; the model will be retrained in the next release.
Averaging the original and reversed option order reduces, but does not remove, sensitivity to option order for three or more options.
About 21% of the training items (13,500 English items) come from an aggregated public corpus whose upstream sources could not be traced item by item; see the technical report.
The test split was consulted during development; its results come from an adaptive process on held-out templates, not from a single untouched final evaluation.
Decisions with serious consequences for people should be reviewed by a person.

Citation

bibtex

@techreport{kinas2026basal,  title       = {basal-1.0: Reliable, Highly Optimized Typed Decisions for Polish},  author      = {Kinas, Remigiusz},  institution = {ai5},  year        = {2026},  type        = {Technical report},  doi         = {10.5281/zenodo.23022986},  url         = {https://doi.org/10.5281/zenodo.23022986}}

License and attribution

Apache-2.0. Fine-tuned from speakleash/Bielik-1.5B-v3.0-Instruct (Apache-2.0).