Measured
Table with columns: Loom Atom, keyword baseline | Loom Atom | keyword baseline |
|---|
| held-out human utterances (1,860) | 96.0% | 64.9% |
| SNIPS — never seen in training (700) | 96.6% | 59.0% |
The second row is the one that matters. SNIPS played no part in training and the score
does not drop — so this is not memorised phrasings. The keyword baseline is a hand-written
list of ~75 tool-ish words scored on the identical splits.
Both test sets are balanced, so chance is 50%.
How small is 22,392 parameters?
Table with columns: parameters, ratio | parameters | ratio |
|---|
| Loom Weave 2 | 59,650,000 | 2,664× |
| Loom Spark 2 | 19,867,008 | 887× |
| Loom Router 1 | 1,435,040 | 64× |
| Loom Atom | 22,392 | 1× |
57 KB. Small enough to embed as a byte array in a header file, and it runs in well under a
millisecond on a CPU.
What it's for
The cheapest useful decision in an agent stack: should this request touch a tool at all?
Put it in front of everything. If it says <none>, you have saved a retrieval call, a
router call, and possibly a large-model call — for the cost of a 57 KB matrix multiply. If
it says <tool>, hand off to something that decides which tool
(Loom Router 1 does that in one token
across 17 routes).
It is not a chat model, a router, or a classifier of intent. It answers one binary
question and nothing else.
Where the floor is
A full ladder was trained, four minutes per rung, identical data:
Table with columns: params, dim, layers, held-out, SNIPS| params | dim | layers | held-out | SNIPS |
|---|
| 86,640 | 48 | 3 | 93.0% | 95.9% |
| 26,976 | 32 | 2 | 95.6% | 93.6% |
| 22,392 | 24 | 2 | 96.0% | |
Depth matters more than width. Narrowing from 24 to 16 dimensions cost about 3 points.
Dropping from two layers to one cost ten. One attention layer can notice keywords; two
can combine a keyword with its context. The floor is a layer count, not a parameter count.
Every rung beats the keyword baseline — even 4,812 parameters, by 17 points.
One honest note: the 86,640-parameter model scores lowest on held-out data because every
rung got the same four minutes, and it completed 3,006 optimiser steps against 24d2L's
10,809. It is under-trained, not worse. Do not read this table as "smaller is better".
Known weakness
Questions about the user personally — "what is my sister's name", "what did I have for
breakfast" — are the hard case. They need no tool (no tool can answer them), but they look
like lookups. Atom gets some right and some wrong; treat <tool> on a first-person
question as unreliable.
Usage — Ollama
ollama run hf.co/textilelabs/Loom-Atom "whats the weather in leeds"
# <tool>
The template and params files in this repo are read automatically. params pins
temperature: 0 and num_predict: 1 — one token, deterministic.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Atom")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Atom").eval()
pair = torch.tensor([tok.convert_tokens_to_ids("<tool>"),
tok.convert_tokens_to_ids("<none>")])
def needs_tool(message: str) -> bool:
p = f"<user>\n{message.strip()}\n<|eot|>\n<loom>\n"
ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
with torch.no_grad():
logits = model(input_ids=ids).logits[0, -1]
return bool(logits[pair].argmax() == 0)
needs_tool("whats the weather in leeds")
needs_tool("i had a rough day")
Prompt format is exact: <user>\n{message}\n<|eot|>\n<loom>\n.
Files
config.json / model.safetensors the model — 67 KB
tokenizer.json / tokenizer_config.json custom BPE tokenizer, 512 tokens
loom-atom-f16.gguf 57 KB, for Ollama / llama.cpp
template / params read automatically by `ollama run hf.co/...`
Modelfile for building locally
Training data
Real human utterances from two openly licensed corpora, relabelled to a single bit:
- MASSIVE — Amazon (CC BY 4.0), derived from SLURP (CC BY 4.0)
- CLINC150 —
clinc/oos-eval (CC BY 3.0)
15,502 utterances, balanced 50/50 by downsampling the majority class. A small procedurally
generated slice written by Textile Labs covers "no tool needed" cases that public assistant
corpora do not contain — chit-chat, ambiguity, and questions only the user can answer.
Both licences require attribution; this section satisfies that and must be kept with any
redistribution.
License
Model: MIT. Training data retains its original licences and attribution as above.