textilelabs

Loom-Weave-3

Why it's smaller than Weave 2

Weave 2 (59.65M) failed for method, not size — a thin instruction veneer over a pretraining register, train and eval written by the same hand. Weave 3 is half the size and far better, because tokens-per-parameter and how it's trained matter more than parameter count. That has precedent: Loom Spark 3 (12.2M) succeeded and beat Spark 2 (19.9M). Bigger is not the goal; behaviour is.

What changed

The reading skill is a real circuit, and it only formed once two things were right:

Table with columns: lever, what went wrong before, fix
lever	what went wrong before	fix
learning rate on the attention matrices	trained 8× too low, so the copy/reading circuit never formed — the model was fluent but couldn't pull "Paris" out of a sentence containing "Paris"	Muon LR set to the family standard (0.025); reading appeared
what the loss grades	it graded every token equally, spending most of its effort on raw web text	loss is masked to the reply tokens; whole conversations packed into each block
how long it trained	over-trained a tiny corpus into memorised, garbled output	short training, kept the best-validation checkpoint
admitting personal limits	learned "I can't know your name" but not "I can't know what you did" — so it hallucinated on other personal questions	added varied personal-unknowable examples; it now declines all of them

The search harness

The model decides a search is needed and writes the query. harness.py does the rest:

searches the model's query and the subject it can see in your question
prefers the real article over lists, films, albums and disambiguation pages
reads the intro first, further only when the intro has no answer of the right kind
strips brackets and pronunciation guides, so real text looks like training text
hands back one sentence, not a paragraph — a model this size misreads paragraphs

Measured behaviour

Same harness and settings a user gets. Run on 2026-09-20.

The acceptance battery, row by row (total ties the best in the Loom family):

Table with columns: row, Loom Weave 3
row	Loom Weave 3
A · says its own name	10/12
B · its own name under rough typing	10/12
C · 5-turn conversation stays on thread	5/5
D · answers from a search result	5/5
E · follow-up answered from the same result	2/5
F · says it looked, after a lookup	5/5
G · never claims a lookup it didn't make	16/16
H · admits what it can't know about you

End to end, everyday questions it had never seen, live Wikipedia, the model writing its own query, scored on the final answer:

Table with columns: set, decided to search, wrote its own query, answer reached the model, answered right
set	decided to search	wrote its own query	answer reached the model	answered right
tuning	20/20	20/20	14/20	11/20
held-out	20/20	20/20	12/20	6/20

It is the first Loom to read live prose above a coin-flip on the tuning set — but read the held-out number honestly: on unfamiliar, messy passages it is right about a third of the time.

Read this before you use it

Every point here was measured.

Held-out live reading is ~30%. "I looked that up" means it searched — not that it read the result correctly. Run harness.py --show and trust the sentence it read.
It is not a calculator. Small sums are out of scope (0/6). Use a tool for arithmetic.
Follow-up questions from the same result are weaker (~40%).
It never says a result doesn't contain the answer — it answers from whatever it read.
It rarely asks a clarifying question on an ambiguous request.
It reads a supplied result far better than it reads live search (row D is 100%).
Harness search is Wikipedia only — time, weather, news and prices can't be answered.
It is a small reader/assistant, not a chat stylist. Voice is plain and brief by design.

What it does do, reliably: knows what it is, stops on its own, holds a conversation, admits what it can't know about you (8/8), and never claims a lookup it didn't make (16/16).

Usage — the harness

bash

python3 harness.py "who wrote dracula"python3 harness.py                          # interactivepython3 harness.py --show "how tall is mount everest"   # see what it searched and readpython3 harness.py --no-tools "who are you"

Stdlib + certifi. Wikipedia needs no API key. Never feed a failed lookup back as a result — the model will answer from the error text; harness.py fails loudly instead.

Usage — Ollama

bash

ollama run hf.co/textilelabs/Loom-Weave-3 "who are you"

template and params are read automatically.

Usage — transformers

python

import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Weave-3")model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Weave-3").eval()eot = tok.convert_tokens_to_ids("<|eot|>") def ask(message, tools=False):    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids    with torch.no_grad():        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]    return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>", "").strip()

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.

How it was built

Table

architecture	Llama — 16 layers × 384d, GQA (6 heads / 2 KV), SwiGLU, RoPE, tied embeddings
parameters	31,469,952
context	1,024
vocabulary	16,384 custom BPE
optimiser	Muon (LR 0.025) on the 2D hidden matrices, AdamW on embeddings and norms
loss	masked to the reply tokens; whole conversations packed per block (FFD)
schedule	warmup → stable → cosine decay (WSD); best-validation checkpoint kept
corpus	173,043 conversations; grounded-reading rows carry real encyclopedic prose

Files

text

config.json / model.safetensors           the modeltokenizer.json / tokenizer_config.json    custom BPE tokenizer, 16,384 tokensloom-weave-3-f16.gguf                      for Ollama / llama.cpp (lookup tags USER_DEFINED)harness.py                                 runnable search harnesstemplate / params                          read automatically by `ollama run hf.co/...`Modelfile                                  for building locallyATTRIBUTION.md                             required credits for the training corpora

Training data

Table with columns: slice, source
slice	source
grounded reading of real prose	SQuAD 2.0 (CC BY-SA 4.0) · Wikipedia (CC BY-SA)
when to reach for a tool	MASSIVE (CC BY 4.0) · CLINC150 (CC BY 3.0)
instruction following	databricks-dolly-15k (CC BY-SA 3.0)
multi-turn dialogue structure	OpenAssistant OASST1 (Apache 2.0)
tokenizer coverage of real prose	HuggingFaceFW/fineweb-edu (ODC-By 1.0)
identity, limits, warmth, attribution	Textile Labs — written for Loom

No language model wrote any training query, and nothing is fine-tuned from anyone's checkpoint. No real user data was used. See ATTRIBUTION.md.

License

Model: MIT. Training data retains its original licences and attribution.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

textilelabs

Model Tree

Base

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Why it's smaller than Weave 2

What changed

The reading skill is a real circuit, and it only formed once two things were right:

Table with columns: lever, what went wrong before, fix
lever	what went wrong before	fix
learning rate on the attention matrices	trained 8× too low, so the copy/reading circuit never formed — the model was fluent but couldn't pull "Paris" out of a sentence containing "Paris"	Muon LR set to the family standard (0.025); reading appeared
what the loss grades	it graded every token equally, spending most of its effort on raw web text	loss is masked to the reply tokens; whole conversations packed into each block
how long it trained	over-trained a tiny corpus into memorised, garbled output	short training, kept the best-validation checkpoint
admitting personal limits	learned "I can't know your name" but not "I can't know what you did" — so it hallucinated on other personal questions	added varied personal-unknowable examples; it now declines all of them

The search harness

The model decides a search is needed and writes the query. harness.py does the rest:

searches the model's query and the subject it can see in your question
prefers the real article over lists, films, albums and disambiguation pages
reads the intro first, further only when the intro has no answer of the right kind
strips brackets and pronunciation guides, so real text looks like training text
hands back one sentence, not a paragraph — a model this size misreads paragraphs

Measured behaviour

Same harness and settings a user gets. Run on 2026-09-20.

The acceptance battery, row by row (total ties the best in the Loom family):

Table with columns: row, Loom Weave 3
row	Loom Weave 3
A · says its own name	10/12
B · its own name under rough typing	10/12
C · 5-turn conversation stays on thread	5/5
D · answers from a search result	5/5
E · follow-up answered from the same result	2/5
F · says it looked, after a lookup	5/5
G · never claims a lookup it didn't make	16/16
H · admits what it can't know about you

End to end, everyday questions it had never seen, live Wikipedia, the model writing its own query, scored on the final answer:

Table with columns: set, decided to search, wrote its own query, answer reached the model, answered right
set	decided to search	wrote its own query	answer reached the model	answered right
tuning	20/20	20/20	14/20	11/20
held-out	20/20	20/20	12/20	6/20

It is the first Loom to read live prose above a coin-flip on the tuning set — but read the held-out number honestly: on unfamiliar, messy passages it is right about a third of the time.

Read this before you use it

Every point here was measured.

Held-out live reading is ~30%. "I looked that up" means it searched — not that it read the result correctly. Run harness.py --show and trust the sentence it read.
It is not a calculator. Small sums are out of scope (0/6). Use a tool for arithmetic.
Follow-up questions from the same result are weaker (~40%).
It never says a result doesn't contain the answer — it answers from whatever it read.
It rarely asks a clarifying question on an ambiguous request.
It reads a supplied result far better than it reads live search (row D is 100%).
Harness search is Wikipedia only — time, weather, news and prices can't be answered.
It is a small reader/assistant, not a chat stylist. Voice is plain and brief by design.

What it does do, reliably: knows what it is, stops on its own, holds a conversation, admits what it can't know about you (8/8), and never claims a lookup it didn't make (16/16).

Usage — the harness

bash

python3 harness.py "who wrote dracula"python3 harness.py                          # interactivepython3 harness.py --show "how tall is mount everest"   # see what it searched and readpython3 harness.py --no-tools "who are you"

Stdlib + certifi. Wikipedia needs no API key. Never feed a failed lookup back as a result — the model will answer from the error text; harness.py fails loudly instead.

Usage — Ollama

bash

ollama run hf.co/textilelabs/Loom-Weave-3 "who are you"

template and params are read automatically.

Usage — transformers

python

import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Weave-3")model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Weave-3").eval()eot = tok.convert_tokens_to_ids("<|eot|>") def ask(message, tools=False):    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids    with torch.no_grad():        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]    return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>", "").strip()

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.

How it was built

Table

architecture	Llama — 16 layers × 384d, GQA (6 heads / 2 KV), SwiGLU, RoPE, tied embeddings
parameters	31,469,952
context	1,024
vocabulary	16,384 custom BPE
optimiser	Muon (LR 0.025) on the 2D hidden matrices, AdamW on embeddings and norms
loss	masked to the reply tokens; whole conversations packed per block (FFD)
schedule	warmup → stable → cosine decay (WSD); best-validation checkpoint kept
corpus	173,043 conversations; grounded-reading rows carry real encyclopedic prose

Files

text

config.json / model.safetensors           the modeltokenizer.json / tokenizer_config.json    custom BPE tokenizer, 16,384 tokensloom-weave-3-f16.gguf                      for Ollama / llama.cpp (lookup tags USER_DEFINED)harness.py                                 runnable search harnesstemplate / params                          read automatically by `ollama run hf.co/...`Modelfile                                  for building locallyATTRIBUTION.md                             required credits for the training corpora

Training data

Table with columns: slice, source
slice	source
grounded reading of real prose	SQuAD 2.0 (CC BY-SA 4.0) · Wikipedia (CC BY-SA)
when to reach for a tool	MASSIVE (CC BY 4.0) · CLINC150 (CC BY 3.0)
instruction following	databricks-dolly-15k (CC BY-SA 3.0)
multi-turn dialogue structure	OpenAssistant OASST1 (Apache 2.0)
tokenizer coverage of real prose	HuggingFaceFW/fineweb-edu (ODC-By 1.0)
identity, limits, warmth, attribution	Textile Labs — written for Loom

No language model wrote any training query, and nothing is fine-tuned from anyone's checkpoint. No real user data was used. See ATTRIBUTION.md.

License

Model: MIT. Training data retains its original licences and attribution.

Loom-Weave-3

README

Why it's smaller than Weave 2

What changed

The search harness

Measured behaviour

Read this before you use it

Usage — the harness

Usage — Ollama

Usage — transformers

How it was built

Files

Training data

License

Explore FriendliAI today

README

Why it's smaller than Weave 2

What changed

The search harness

Measured behaviour

Read this before you use it

Usage — the harness

Usage — Ollama

Usage — transformers

How it was built

Files

Training data

License