GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Ismantic
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Ismantic
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Chat (v9e2) — single-stage post-training from Summer-0.5B-S0 (not from
Summer-0.5B-S1; the parallel-data anneal is a separate branch for the
translation downstream, unrelated to becoming a general chat model).
This follows nanochat's current recipe (chat_sft.py, commit 1ddaad1
onward): no separate mid-training stage. An earlier version of this
project's chat line used a two-stage midtrain-then-SFT design copied from an
older nanochat snapshot; that stage was dropped once we confirmed nanochat
itself had merged it into a single SFT pass, relying on a large enough data
mixture to teach the special tokens (<user>/<assistant>/<end>) instead
of a dedicated full-loss warm-up stage.
chat_v9 mixture, 305,187 rows / 625M tokens — the same
chat_full mixture as the previous (v8) release (SmolTalk full 460K,
MMLU-aux x1, ARC-Easy/Challenge, GSM8K x4, spelling tasks, Chinese data
scaled to keep row-share constant), plus three targeted additions made
after manual testing of v8 surfaced specific, reproducible failures:
Summer-0.5B-S0
(monolingual), so the chat model had never seen a translation
instruction; asking it to translate produced fabricated, off-topic text
rather than an attempt.init_lr_frac=0.8, spread across the full
training run below (not decayed-then-restarted).chat_v9 data (v9 itself was
1 epoch, 4,768 steps). At 524M params / 625M post-training tokens
(~1.2 tokens/param), there was headroom to see the data twice without
the val loss plateauing — it kept dropping past where the 1-epoch run
ended (1.2459 → 1.1932 final).Every input must start with <bos>, same as Summer-0.5B-S0.
This is a 14.6B + 625M chat-token model. For scale: Qwen3-0.6B-Base saw 36T tokens —
about 2,700x more. Treat the numbers below as what that budget buys, not as a
competitive result.
| nanochat d20 | v9 (1 epoch) | v9e2 (this release, 2 epochs) | |
|---|---|---|---|
| ARC-Easy | 0.4033 | 0.4440 | 0.4882 |
| ARC-Challenge | — | 0.3584 | 0.3848 |
| MMLU | 0.3232 | 0.3296 | 0.3455 |
| C-Eval | — (d20 has no Chinese) | 0.3646 |
A second epoch over the identical data raised every letter-MC number by 0.016-0.044 — a bigger jump than the v8→v9 data changes produced, and well outside sampling noise (e.g. MMLU's 14,042-item standard error is ~0.004; the observed gap is 4x that). Makes sense: repeated exposure reinforces the "pick a letter correctly" multiple-choice skill more directly than adding new data varieties does. Still beats d20 on every comparable metric.
repetition_penalty=1.15, not greedy)Greedy decoding (temperature=0) is only the scoring protocol used above —
it is not what this model is served with. repetition_penalty=1.15 was
found empirically to be the deployment sweet spot (see this project's
prepare/stoprate.py for the full derivation); nanochat's own interactive
chat_cli.py also does not use greedy decoding by default.
| en natural-stop | en long-form success | zh natural-stop | zh long-form success | |
|---|---|---|---|---|
| v9 (1 epoch) | 100% | 68% | 84% | 25% |
| v9e2 (2 epochs) | 100% | 74% | 82% | 24% |
English long-form success improved (68%→74%); Chinese is flat within noise
(n=100 per language, standard error ~4-5 points). Not a clean sweep, but no
regression either.
"Long-form success" = >=150 tokens, repetition ratio <0.2, stopped naturally.
Chinese open-ended creative writing (poems, stories) is still the weakest
spot — it is more prone to repetition collapse than factual Q&A, a known,
unresolved limitation (root cause not fully identified; greedy decoding is
the primary suspect, not the training recipe, per this project's own
ablations — see docs/POSTTRAIN.md).
v9 fixed identity and translation but not one specific failure mode (responding to a remark rather than a question); the second epoch re-sees the same data, so this was re-tested rather than assumed fixed:
| Scenario | v9 (1 epoch) | v9e2 (2 epochs) |
|---|---|---|
| "Who are you?" | consistently answers "Summer-0.5B" | still consistent (now also states the parameter count unprompted) |
| "Translate to English: 我是中国人" | "I am Chinese." — correct | still correct |
| Given a bubble-sort explanation, then told "this looks like a pretty good model" (a remark, not a question) | invented an unrelated "time-series analysis model" tangent | still fails — this time it stayed on the bubble-sort topic but misread the remark as a request to critique/optimize the algorithm, rather than acknowledging the compliment |
Repeating the same 8,000 rows of WildChat data a second time did not fix
the remaining gap — as expected. This specific pattern (a bare compliment,
no request attached) is naturally rare even in real conversation logs
(WildChat mining found roughly 1% of candidate turns match it), so repetition
reinforces already-thin signal rather than creating new signal. A synthetic,
hand-written data source targeting this exact shape (analogous to how the
Identity task itself is hand-written, not mined) is the planned next step,
not included in this release.
This is a 524M-parameter model trained on 14.6B pretraining + 625M
post-training tokens — for scale, Qwen3-0.6B-Base saw 36T tokens, about
2,700x more. Do not expect factual reliability. Manual testing surfaces
confident-sounding but fabricated numbers on things like population figures,
country counts, or basic arithmetic ("how many days in a week"). This tracks
with what the letter-MC benchmark measures (does it follow instructions, pick
a reasonable-sounding answer) rather than what it does not measure (is the
answer actually correct) — the two are different questions, and this model
was optimized against the former. Self-identification and translation
requests are handled reliably; responding sensibly to a remark that isn't
phrased as a question is still unreliable.
The tokenizer is a compiled C++ extension, not loadable by
AutoTokenizer. The release ships tokenizer.py and example_load.py:
bash
pip install git+https://github.com/Ismantic/PieceTokenizerpython example_load.py
The model code (model.py, checkpoint.py) is bundled too — the package
depends only on torch plus the tokenizer extension, not on transformers.
Full pipeline, data mixes and every design decision (including the mistakes)
are documented in https://github.com/Ismantic/Summer. Notably
docs/WHY.md records why fp32 master weights are mandatory, why the learning
rate schedule is WSD rather than cosine, and what the vocabulary swap cost.
Apache-2.0. Training corpora are public datasets (FineWeb-Edu, Cosmopedia,
CCI3-HQ, SkyPile, WMT19, OPUS-100 and others; see data/source.py upstream).
Please observe their respective licenses.
| Format-following (free_argmax_is_letter) | ~1.0 | 0.99-1.00 | 0.99-1.00 |