The headline result
Every prior full-size Loom release (v1, 1.5, 1.8) only learned <|endoftext|> at the
very end of a whole training document — mid-conversation, nothing told the model a turn
had ended, so on a runtime without exactly the right stop-token setup it would keep
generating and invent your next message itself. This happened to the founder testing
Loom Spark 1.8 in Ollama the day it shipped.
This model's curriculum was rebuilt so <|endoftext|> follows every single reply,
not just document end. Verified: 442,333 / 442,333 model turns in the training corpus
end in EOS. Tested with zero configuration — Ollama's stock chat template, no
Modelfile, no stop tokens set by hand: 0 self-dialogue turns, both through the
default chat endpoint and through the real agent harness with live web search. It stops
because it learned to.
How it compares to 1.5 Flash
Both were trained on the identical 70MB curriculum and tokenizer. The only difference is
capacity (128d×4L×4H vs 192d×4L×4H) and training budget, matched by step count
(2,625 vs 2,666 steps) rather than wall-clock minutes — an earlier version of this
model was trained for equal minutes instead of equal steps, which under-trained it
relative to 1.5 Flash despite being bigger. That run was discarded; these are the
numbers from the properly step-matched retrain.
Table with columns: 1.5 Flash, 1.8 Flash | 1.5 Flash | 1.8 Flash |
|---|
| Params | 1.35M | 2.62M |
| Val loss | 0.3544 | 0.3332 |
| Identity probes leaking | 0/12 | 0/12 |
| Offline fact leak (raw model) | 18/30 | 15/30 |
| Clean online lookups | 5/10 | 6/10 |
| Self-terminates, zero config | ✅ | ✅ |
Honest limitations — read this first
Still a very small model. Expect wrong or garbled answers to most factual questions —
there's more room here than 1.5 Flash for identity and structure to hold up, but a real
fact-core still doesn't fit. What holds up well: identity, restraint, and emotional
register — 0/12 identity probes leaked, matching the best of the full-size models.
<tools:off>
<tools:off><user> who are you
<loom>
No trailing space after <loom>. With tools on, a reply may end in
<lookup>query</lookup><|endoftext|>; your harness splices in <result>…</result>
before continuing.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Spark-1.8-Flash")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Spark-1.8-Flash")
prompt = "<tools:off>\n<tools:off><user> who are you\n<loom>"
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids
out = model.generate(ids, max_new_tokens=100, do_sample=True,
temperature=0.8, top_k=50, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False))
Usage — Ollama
ollama create loom-spark-1.8-flash -f ollama/Modelfile
ollama run loom-spark-1.8-flash
Safe to run with no Modelfile at all — ollama run hf.co/textilelabs/Loom-Spark-1.8-Flash
will not talk to itself, though output quality is better with the correct template.
Usage — the agent harness
pip install ./harness
loom-chat --model textilelabs/Loom-Spark-1.8-Flash
Harness v0.2.2+ required — auto-detects this model's format via an explicit flag in
config.json rather than guessing from parameter count.
Training
- Hardware: CPU-only Dell OptiPlex 9020, i5-4690, 4 cores, no GPU
- 2,666 steps, batch 32 × 256 tokens, step-matched to 1.5 Flash's budget
- Corpus: same 70MB curriculum as Loom Spark 1.8, EOS after every model turn
- Final validation loss: 0.3332
- Architecture: 192d × 4 layers × 4 heads, vocab 4096 (fresh BPE, shared with 1.5 Flash,
not shared with any full-size Loom generation)
Files
config.json / model.safetensors transformers weights
tokenizer.json / tokenizer_config.json 4096-token custom BPE (Flash-specific)
loom-spark-1.8-flash-f32.gguf GGUF for llama.cpp / Ollama
ollama/Modelfile correct template + stop tokens
harness/ agent harness v0.2.2 with web search
evaluation/ acceptance logs, comparison vs 1.5 Flash
License
MIT. See LICENSE.