labqoat

squad-0.8b

README

License: apache-2.0

Results

All numbers measured with the same benchmark client; each model served by its own runtime (this repo's qev_fast engine, per-question requests). basal-omni-pico (Remek/basal-omni-pico) was served by basal 2.4.0, fast mode.

Table with columns: squad-0.8b (previous), squad-0.8b (this revision), basal-omni-pico
	squad-0.8b (previous)	squad-0.8b (this revision)	basal-omni-pico
Benchmark Heaven (public, 231)	77.1	76.6	75.8
JevExtra	71.3	71.7	64.6
JevSpatial	75.8	79.7	68.4
Rules-7	63.8	74.1	69.5
JevBench accuracy	84.9	86.4	84.0
JevBench coherence	70.6	76.4	74.2
KLEJ (Polish, 7 tasks, 500 test items each)	70.9 (trunk only)	73.4	75.3
games_bench (9 unseen games, closed loop; 0 = random, 100 = planner)	11.3	10.4	7.5
e2e on 8 real web apps (61 tests × 3 runs)	10 / 183	34 / 183	–

Speed on one NVIDIA L40S (Benchmark Heaven requests, one question each, after warm-up):

Table with columns: this revision, basal-omni-pico (fast), basal-omni-pico (eager)
	this revision	basal-omni-pico (fast)	basal-omni-pico (eager)
single request, median / p95	3.7 / 30.5 ms	6.0 / 31.5 ms	50.4 / 101.3 ms
16 requests at once	115 decisions/s	94 decisions/s	33 decisions/s
GPU memory	6.3 GB	–	–

Read these honestly:

Benchmark Heaven moves by about ±1 point between runs (one item is 0.43 points); its hard tier (111 items) is around 57% for every model here.
Rules-7: part of the gain comes from procedurally generated rule-application data whose layout (the policy inside the question) resembles Rules-7's. No benchmark text was used; labels are computed by code. Sub-scores vary across training seeds.
The e2e expert is used only when the router gives it at least 35% (a serving setting chosen while looking at Benchmark Heaven and the real-app tests). It still fails the TodoMVC apps (it clicks instead of pressing Enter).
games_bench is our own closed-loop benchmark of games never trained on; all models are close to random when a game is shown as a raw grid, coordinates or prose.
Image input was not re-evaluated for this revision (the trunk changed; the vision tower is unchanged).
Benchmark scores were used for reporting; checkpoints were chosen on our own validation data, except the serving threshold above.

How it works

Trunk: a full fine-tune of Qwen3.5-0.8B-Base (24 layers, Gated DeltaNet + gated attention). It continues the previous qev-0.8b trunk on procedurally generated, exactly labelled data: consistency twins (the same decision asked in rephrased, reordered, merged or complementary forms, trained with a consistency loss), rule application with natural "up to / below" semantics, graded similarity, plus replay of the old data. Final weights are a weight average of three runs. No LLM-written text is in the trunk.
Experts: rank-64 LoRA on gate_proj, up_proj, down_proj of layers 12–23, y = W x + 2.0 · B A x, stored as 2-bit rows (specialists/slots_2bit.safetensors). Slot 1 games, 2 e2e, 3 multilingual, 4 documents. Trained by reinforcement learning only (closed-loop games, simulated web apps, decision tasks with a proper-scoring reward), trunk frozen. The multilingual and documents experts learned from data written and cross-checked by two LLMs.
Router: LayerNorm → Linear(1024, 256) → GELU → Linear(256, 9) on the mean RMS-normalised residual after layer 5 (specialists/router.safetensors), top-1 of {trunk, games, e2e, multilingual, documents}. Trained on measured outcomes (log-probability of the right answer) on validation rows no expert saw, so it only calls an expert where it helps. router.min_p_slots in the manifest holds per-expert confidence thresholds (e2e: 0.35).

Quickstart (NVIDIA GPU)

pip install -r requirements.txt -r requirements-cuda.txt

python

from qev import QEV m = QEV.from_pretrained("labqoat/squad-0.8b", specialists=True)     # router + experts (fast CUDA engine)m.decide(state, questions, per_question=True)                        # System One answersm.last_routes                                                        # e.g. ["games"]

python qev_server.py --model labqoat/squad-0.8b --specialists     # POST http://127.0.0.1:8017/v1/systemone

Without specialists=True you get the trunk alone (not the old qev-0.8b: the trunk changed in this revision; pin the previous revision 2043fda for it). Experts need the fast CUDA engine (requirements-cuda.txt).

On a Mac or CPU

MLX (labqoat/squad-0.8b-MLX): router + 4 experts, about 50 ms per decision on an M4 Pro; same top answer as this runtime on 100% of 387 test questions (bf16).
GGUF (labqoat/squad-0.8b-GGUF): llama.cpp b11786 or newer (/v1/systemone), one LoRA file per expert, qev_gguf.py for routing; same route on 333/333 requests and the same top answer on 99.5% of questions (Q8_0, Metal).

Files

model.safetensors (new trunk; embeddings, head and vision tower as before) and tokenizer / config files · specialists/ (manifest, router, 2-bit experts) · qev.py, qev_server.py, qev_fast/ (runner, server, CUDA engine with experts) · qev_image.py, qev_image_server.py (transformers image path, trunk only) · SHA256SUMS.

Limitations

Text decisions only were evaluated for this revision. The e2e expert is far from reliable on real apps (34 of 183 real-app test runs pass). Polish sentiment and QA (KLEJ) are behind basal-omni-pico. Probabilities are calibrated with the previous version's temperatures; check them on your own data before using them as error rates.

License: Apache-2.0.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

labqoat

Model Tree

Base

Qwen/Qwen3.5-0.8B-Base

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality