ramput-labs

den-qwen3.5-4b

README

License: apache-2.0

Use

bash

git clone https://github.com/ramput-labs/den && cd den && make setupecho '{"state": "I was charged twice for my order.", "questions": {"team": {"type": "choice",  "instructions": "Which team handles this?", "criteria": {"billing": "payments", "tech": "bugs"}}}}' \  | uv run den predict --run hf:ramput-labs/den-qwen3.5-4b@v1

It runs on Apple Silicon (MLX), NVIDIA (CUDA) and CPU. The first call downloads the run into the Hub cache. Always load v1 pinned (@v1): its merged weights live in the v1 tag only, and main holds the adapter and head.

Results

At T, over hard-labelled questions: accuracy, NLL, Brier score, expected calibration error, and coverage at 5% error (the share of questions answerable at <= 5% error, most confident first). eval.json also has accuracy per question type and, for score questions, the expected level's mean absolute error. probes is mostly unknowable items, scored on confidence, not accuracy.

Table with columns: file, questions, accuracy, nll, brier, ece, coverage at 5% error
file	questions	accuracy	nll	brier	ece	coverage at 5% error
`dev/binding.jsonl`	560	0.9732	0.0742	0.0423	0.0143	1.000
`dev/breadth.jsonl`	3075	0.7005	0.7095	0.3855	0.0353	0.287
`dev/core.jsonl`	1468	0.8760	0.3755	0.1853

Calibration

Table with columns: split, at, accuracy, nll, ece
split	at	accuracy	nll	ece
calibration (fit)	T = 1	0.8145	0.7304	0.1078
calibration (fit)	T = 2.180	0.8145	0.5155	0.0099
dev	T = 1	0.8160	0.5963	0.0671
dev

Lowest dev NLL (T = 1, 600 dev sample questions): step 1600 of 2657, nll 0.5969, accuracy 0.8167: best/.

Minimal pairs

Table with columns: file, pairs, pair accuracy, true option present, true option removed
file	pairs	pair accuracy	true option present	true option removed
`dev/core.jsonl+pairs`	628	0.7850	0.8392	0.8854
`dev/documents.jsonl+pairs`	920	0.6837	0.7272	0.9109
`dev/skills.jsonl+pairs`	824

Integrity

12 of 12 checks passed (integrity.json); 23 model files with their sha256.

Training data licences

Table with columns: class, records
class	records
open	23261
share-alike	5033
non-commercial	1623
unspecified	4903
restricted	3300

Some training records come from sources whose terms limit their use (non-commercial, unspecified, restricted; see data/LICENSES.md in the data repo). The weights carry the base model's licence; whether those data terms reach a model trained on them is a legal question this card does not settle. den train --max-licence share-alike trains only on data whose terms allow commercial use.

Files

merged/ (in the v1 tag only): the LoRA folded into bf16 weights, exactly as evaluated. It was merged on an H100 with TF32, so it can't be rebuilt bit for bit from the adapter; den loads it from @v1. From v2 on, releases ship the adapter and head only, and den rebuilds merged/ on first load, checked against integrity.json.
head.safetensors, head.json: the pointer head's weights, and its config (kind, size, the backbone's hidden size) and T. No pickle: the whole repo is safetensors and JSON.
adapter_model.safetensors, adapter_config.json: the LoRA alone, for use on top of the base.
run.json, training_config.json, eval.json: how it was trained (loss log, dev curve, calibration) and measured.

Training

Each stage continued from the one before it (Kev-4B's recipe, with Kev's none-of-the-above augmentation), on NVIDIA H100 80GB HBM3. The last stage's temperature ships.

Table with columns: stage, data, epochs, lr, steps, minutes, T
stage	data	epochs	lr	steps	minutes	T
`1-base`	train/core.jsonl	2	5e-05	4450	49.1	2.186
`2-dates`	train/dates-unknowable.jsonl + 2000 replayed	1	2e-05

Code 1476410d123e10d7a2681bebc7a0dc3559cdffc2+dirty; torch 2.14.1, transformers 5.17.0, peft 0.21.2, accelerate 1.15.0, tokenizers 0.23.2, safetensors 0.8.0, huggingface-hub 1.33.0, unsloth 2026.10.3, unsloth_zoo 2026.10.3, flash-linear-attention 0.5.2, den 0.1.0.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

ramput-labs

Model Tree

Base

Qwen/Qwen3.5-4B-Base

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality