GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
LoNebula
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
LoNebula
Model Tree
Input Modalities
Output Modalities
Supported Functionality
All four arms of the pre-registered ablation are published, so the comparison can be reproduced and inspected.
| Repository | Arm | What it is |
|---|---|---|
| LoNebula/Nebula-1-preview | A0 | This is arm A0 of our pre-registered ablation: next-token prediction only, without D1 (self-definition anchoring). |
| LoNebula/Nebula-1-ablation-a1 | A1 | This is arm A1 of our pre-registered ablation: it was trained with D1 (self-definition anchoring) on top of A0. During training the model reads Wiktionary definitions of rare tokens, and an auxiliary loss pulls those tokens' embeddings toward the model's own pooled hidden state over the definition. |
| LoNebula/Nebula-1-ablation-a2 | A2 | This is arm A2 of our pre-registered ablation: A0 plus the same Wiktionary definitions, inserted as plain text documents (<word>: <definition>) at the same total token cost as A1's definition reads. It is the control that separates the D1 mechanism from simply reading dictionary text. |
| LoNebula/Nebula-1-ablation-a0-seed1235 | A0' | This is arm A0' of our pre-registered ablation: the A0 baseline trained again with seed 1235 (initialization and data order), so that the difference between A0 and A0' could be used as the seed-noise level of the decision rule. |
Every number below was measured by us with lm-evaluation-harness 0.4.13, under one fixed setting (same seed, dtype and batch size for every model). We did not copy any official number. Comparison models are public base models of 1B parameters or fewer. Values are percentages; the best value per column is bold.
| Model | Params | MMLU | ARC-e | ARC-c | HellaSwag | WinoGrande | PIQA | LAMBADA | GSM8K | MMLU-ja | JComQA | JNLI | XWinograd-ja | Belebele-ja | MGSM-ja | Avg (en) | Avg (ja) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Nebula-1-preview | 122M | 25.0 | 38.4 | 23.6 | 27.5 | 51.0 | 58.2 | 17.7 | 0.2 | 24.8 | 42.9 | 45.5 | 54.7 | 22.7 | 1.6 | 30.2 | 32.0 |
| HuggingFaceTB/SmolLM2-135M | 135M | 25.2 | 58.4 | 30.0 | 43.0 | 53.0 | 68.3 | 42.9 | 1.4 | 26.4 | 23.1 | 44.7 | 51.6 | 22.9 | 1.2 | 40.3 | 28.3 |
| facebook/MobileLLM-R1-140M-base | 140M | 23.8 | 50.0 | 24.7 | 34.1 | 50.7 | 62.9 | 31.2 | 19.0 | 25.3 | 22.0 | 49.9 | 52.2 | 21.4 | 1.6 | 37.1 | 28.7 |
| llm-jp/llm-jp-3-150m | 152M | 25.8 | 38.3 | 24.7 | 31.6 | 50.0 | 60.9 | 28.2 | 0.2 | 25.9 | 56.7 | 46.7 | 55.0 | 24.2 | 1.2 | 32.5 | 35.0 |
| google/gemma-3-270m | 268M | 26.5 | 57.2 | 28.1 | 41.4 | 53.6 | 67.8 | 42.9 | 1.3 | 27.3 | 46.1 | 38.4 | 55.2 | 23.1 | 1.2 | 39.8 | 31.9 |
| LiquidAI/LFM2-350M | 354M | 43.1 | 66.7 | 39.6 | 49.0 | 56.7 | 69.0 | 39.8 | 34.0 | 36.8 | 54.6 | 36.8 | 54.8 | 39.8 | 0.0 | 49.8 | 37.1 |
| facebook/MobileLLM-R1-360M-base | 359M | 26.5 | 61.2 | 34.4 | 44.1 | 55.2 | 69.0 | 39.4 | 41.6 | 24.0 | 29.2 | 55.0 | 53.4 | 27.6 | 0.0 | 46.4 | 31.5 |
| HuggingFaceTB/SmolLM2-360M | 362M | 25.1 | 68.1 | 38.1 | 56.2 | 59.7 | 72.0 | 53.2 | 4.2 | 26.0 | 26.6 | 49.3 | 51.5 | 28.2 | 2.0 | 47.1 | 30.6 |
| EleutherAI/pythia-410m | 405M | 26.6 | 45.2 | 24.3 | 40.4 | 53.0 | 67.0 | 47.5 | 1.2 | 26.3 | 26.1 | 37.4 | 52.8 | 23.1 | 0.8 | 38.2 | 27.8 |
| llm-jp/llm-jp-3-440m | 447M | 24.8 | 45.1 | 24.3 | 42.5 | 52.0 | 66.7 | 39.5 | 1.3 | 24.9 | 59.8 | 40.3 | 58.3 | 22.1 | 0.8 | 37.0 | 34.4 |
| Qwen/Qwen2.5-0.5B | 494M | 47.6 | 58.8 | 31.9 | 52.1 | 56.7 | 70.0 | 52.0 | 34.1 | 37.5 | 35.8 | 39.8 | 58.3 | 33.4 | 0.0 | 50.4 | 34.1 |
| LiquidAI/LFM2-700M | 742M | 49.4 | 71.0 | 46.9 | 57.5 | 59.1 | 71.0 | 45.1 | 46.4 | 42.6 | 65.1 | 29.8 | 58.9 | 44.2 | 12.4 | 55.8 | 42.2 |
| sbintuitions/sarashina2.2-0.5b | 793M | 27.3 | 37.0 | 26.7 | 35.8 | 54.8 | 58.1 | 35.1 | 19.4 | 27.8 | 72.7 | 30.3 | 66.0 | 27.3 | 23.6 | 36.8 | 41.3 |
Not measured: Qwen/Qwen3-0.6B-Base, facebook/MobileLLM-R1-950M-base, llm-jp/llm-jp-3-980m, google/gemma-3-1b-pt
On the English average (30.2) this model ranks 13 of 13. On the Japanese average (32.0) it ranks 7 of 13.
| Task | Language | lm-eval task | Shots | Metric |
|---|---|---|---|---|
| MMLU | en | mmlu | 5 | acc |
| ARC-e | en | arc_easy | 0 | acc_norm |
| ARC-c | en | arc_challenge | 0 | acc_norm |
| HellaSwag | en | hellaswag | 0 | acc_norm |
| WinoGrande | en | winogrande | 0 | acc |
| PIQA | en | piqa | 0 | acc_norm |
| LAMBADA | en | lambada_openai | 0 | acc |
| GSM8K | en | gsm8k | 5 | exact_match |
| MMLU-ja | ja | global_mmlu_full_ja | 5 | acc |
| JComQA | ja | nebula_jcommonsenseqa | 3 | acc |
| JNLI | ja | nebula_jnli | 3 | acc |
| XWinograd-ja | ja | ja_leaderboard_xwinograd | 0 | acc |
| Belebele-ja | ja | belebele_jpn_Jpan | 0 | acc |
| MGSM-ja | ja | ja_leaderboard_mgsm | 5 | acc |
Avg (en) and Avg (ja) are plain means of the task scores of that language. Per-task results are in eval/.
All text was filtered, tokenized with nebula-64k and checked against the evaluation suite (see below).
| Source | Language | License | Share of tokens |
|---|---|---|---|
| HuggingFaceFW/fineweb-edu (sample-10BT) | en | ODC-BY-1.0 | 40% |
| HuggingFaceTB/finemath (finemath-4plus) | en | ODC-BY-1.0 | 5% |
| wikimedia/wikipedia (20231101.en) | en | CC-BY-SA-3.0 | 5% |
| hotchpotch/fineweb-2-edu-japanese (sample_10BT) | ja | ODC-BY-1.0 | 40% |
| wikimedia/wikipedia (20231101.ja) | ja | CC-BY-SA-3.0 | 10% |
Decontamination. Text is normalized (NFKC, lowercase, no whitespace, punctuation or control characters). A training document is dropped if any 50-character window of it matches a window of the evaluation documents, so an overlap of 75 normalized characters or more is always caught.
D1 reads a rare token's dictionary definition with the model that is being trained, pools the hidden states, stops the gradient, and pulls the token's embedding rows toward that vector. In our searches, run on 16 October 2026, we did not find the same method. This is not proof that nobody has done it. The nearest work is DefinitionEMB (2408.01308, applied after pretraining with a separate encoder) and Token Distillation (2505.20133, applied after pretraining to initialize new tokens).
python
from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("LoNebula/Nebula-1-preview")model = AutoModelForCausalLM.from_pretrained("LoNebula/Nebula-1-preview")inputs = tok("The capital of Japan is", return_tensors="pt")out = model.generate(**inputs, max_new_tokens=32, do_sample=False)print(tok.decode(out[0], skip_special_tokens=True))The beginning-of-sequence and end-of-sequence token is <|endoftext|> (id 0). The context length is 2048.
from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("LoNebula/Nebula-1-preview")model = AutoModelForCausalLM.from_pretrained("LoNebula/Nebula-1-preview")inputs = tok("The capital of Japan is", return_tensors="pt")out = model.generate(**inputs, max_new_tokens=32, do_sample=False)print(tok.decode(out[0], skip_special_tokens=True))