PelaAI

KnowLine-4B-Gen1

README

License: apache-2.0

What it does

You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options:

one forward pass per question, with no generated text;
the output is the probability of each option's label token;
existing Jev clients only need a new base URL.

Table

Base model	Qwen/Qwen3.5-4B (Apache-2.0)
Training	LoRA SFT (rank 32, alpha 64) on the language model, one epoch over about 713k rows. This release is step 5,650 of 5,655, the lowest validation loss.
Release format	LoRA merged into the base; bf16 weights. The vision tower and MTP head are unchanged.
Languages	English, Simplified Chinese, Traditional Chinese

Quickstart

bash

pip install "sglang==0.5.21" "transformers==5.12.1" requestsbash serve_knowline.sh PelaAI/KnowLine-4B-Gen1 0 8080     # SGLang (FP8 at load) on :9080 + /v1/systemone on :8080curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{  "model": "m",  "state": "Customer: my order arrived broken, I want my money back.",  "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},                "tone": {"type": "choice", "instructions": "Customer tone?",                         "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'

Without SGLang: python knowline_server.py --model PelaAI/KnowLine-4B-Gen1 --backend hf --port 8080 uses transformers only. It is slower and runs bf16.
Front end: knowline_server.py is a single file and needs only transformers and requests.
Full settings: the exact settings of our Decision Index runs are in INFERENCE.md.

Comparison

Decision Index 0.3, public suite. The full 0.3 score adds private tests that only the maintainers run (0.5 same-skill, 0.3 new-domain); ours is not available yet.

Table with columns: model, size, DI 0.3 public, DI 0.3 full, source
model	size	DI 0.3 public	DI 0.3 full	source
KnowLine-4B-Gen1	4B	60.47	not yet scored	self-run, official kit
Clef	27B	61.71	53.08	board
Jev 1.13	(API)	57.96	60.11	board

Our public-suite scores by area:

Table with columns: area, KnowLine-4B-Gen1 (0.3 public), Jev 1.13 (0.2.1)
area	KnowLine-4B-Gen1 (0.3 public)	Jev 1.13 (0.2.1)
Knowledge & reasoning	37.7	51.4
Language	60.8	62.0
Retrieval & routing	70.9	55.4
Tools & agents	86.8	75.1
Arts & taste	49.5	37.7

Releases

The model is trained in a self-evolving loop, and new versions will follow.

Table with columns: model, date, DI 0.3 public, DI 0.2.1, golden held-out (en / zh-Hans / zh-Hant), notes
model	date	DI 0.3 public	DI 0.2.1	golden held-out (en / zh-Hans / zh-Hant)	notes
KnowLine-4B-Gen4	2026-10-10	64.90	—	69.7 / 70.1 / 65.1	fourth release
KnowLine-4B-Gen3	2026-10-09	63.11	—

Evaluation

All results are self-run and not verified by a third party.

Held-out and Chinese evaluations

Table with columns: suite, accuracy
suite	accuracy
In-house evaluation set, English	68.7
In-house evaluation set, Simplified Chinese	69.9
In-house evaluation set, Traditional Chinese	64.1
C-Eval (4 categories, macro)	77.9
Open-Jev 1.1 test / OOD	87.4 / 86.6

Web operation (Mind2Web official test splits, evaluation only)

Table with columns: split, element selection, operation (balanced)
split	element selection	operation (balanced)
test_task (websites seen in training, new tasks)	93.1	96.6
test_website (new websites)	90.7	96.5
test_domain (new domains)	91.5	98.5

So far this is only used to explore the model in RPA-style automation and to check that it generalises to some degree.
The task is to pick the target element among it and up to 5 other candidates sampled from the page. This is easier than the original Mind2Web protocol, so do not compare it with the Mind2Web leaderboard.

Game harness (KOF '98)

Setup: single-bout character-mirror round robin, 11 players, 18 games per pair (990 games), argmax actions.
Result: win rate 0.883 [0.83, 0.92], Elo 1910, tied for first of 11 with another internal checkpoint.
Head-to-head: 17-0-1 against Jev 1.13, 15-3 against StartLux-Decision-4B, 18-0 against Clef-Flash.
Caveat: the policy relies heavily on one move, a 623C anti-air uppercut used about 55% of the time. This is a measured result in this harness, not general fighting-game skill.

Disclosures

Decision Index training data: about 25% of the data is in Decision Index format, that is, synthetic items written in the benchmarks' request formats. No Decision Index test item is included.
Decontamination:
- Every component text (state, instructions, option texts) was checked against every Decision Index row and all of our evaluation sets.
- Rows that are identical, or share any run of 50 consecutive characters, were removed.
Game data: labels come from simulator rollouts or engines. Seeds and start states are disjoint from the evaluation states.
No model outputs as labels: no output of Jev or any other decision model was used as a training label.
Teacher-labelled synthetic data: LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but not all were checked by a human.

Limitations

Knowledge-heavy reasoning: weaker than larger models. The knowledge area is 37.7 (0.3); MMLU-Pro, GPQA and HLE are close to the base model.
Math: answered without reasoning. The rebuilt 0.3 GSM8K scores 29.5.
Prompt injection: an instruction planted in the state changes the answer about 9% of the time on our injection set. Keep untrusted text clearly delimited.
Calibration: ECE is about 0.05-0.06 on choice and yes/no questions and about 0.12 on score questions. For score questions, fit a temperature on your own data. On the Decision Index 0.3 public suite, computed the board's way, ECE is about 0.08-0.09 and 3.3% of answers are wrong at ≥95% confidence (board median 1.3%): the model is overconfident overall.
Private tests: part of our public-suite advantage comes from adapting to the question formats, so the 0.3 private tests may score lower.

Citation

bibtex

@misc{knowline4bgen1,  title  = {KnowLine-4B-Gen1: a 4B decision model},  author = {PelaAI},  year   = {2026},  url    = {https://huggingface.co/PelaAI/KnowLine-4B-Gen1}}

License

Weights: Apache-2.0, the same as the base model.
Code: knowline_server.py is MIT (see the file header).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

PelaAI

Model Tree

Base

Qwen/Qwen3.5-4B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality