GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
CodeSoft
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
CodeSoft
Model Tree
Input Modalities
Output Modalities
Supported Functionality
From-scratch ~25M-parameter Qwen2-style decoder LM trained in under 4h on a single RTX 5060 Ti (16GB).
| Params | 25,185,920 (~87% non-embedding) |
| Layers / hidden | 14 / 384 |
| Attention | GQA 6 heads / 2 KV heads, RoPE θ=100k |
| FFN | 1024 (SwiGLU) |
| Context | 4096 |
| Vocab | 8,192 custom byte-level BPE (tied embeddings) |
| Precision | bf16 |
0.8B-token weighted mix: fineweb-edu 70% / infiwebmath 10% / DCLM-baseline 20%, block-shuffled. ~3000 steps at 262,144 tok/step, cosine LR, 8-bit AdamW.
| Task | n | Random | acc | acc_norm |
|---|---|---|---|---|
| HellaSwag | 10,042 | 25% | 26.52 ±0.44 | 26.12 ±0.44 |
| ARC-easy | 2,376 | ~25% | 29.50 ±0.94 | 29.59 ±0.94 |
| ARC-challenge | 1,172 | ~25% | 17.66 ±1.11 | 22.95 ±1.23 |
Notes:
AxiomicLabs/Arithmark-3.0) is the strongest relative result
(+7.9 pts over random), consistent with the math share of the pretraining mix.python
from transformers import AutoModelForCausalLM, AutoTokenizerrepo = "CodeSoft/sorbet-25m"model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16").to("cuda")tok = AutoTokenizer.from_pretrained(repo, subfolder="tokenizer")ids = tok("Once upon a time", return_tensors="pt").input_ids.cuda()print(tok.decode(model.generate(ids, max_new_tokens=64)[0]))
Expect shallow world knowledge and weak performance on knowledge-heavy benchmarks due to the model's small parameter count and limited training budget.
Apache-2.0.
| PIQA | 1,838 | 50% | 54.46 ±1.16 | 53.43 ±1.16 |
| ArithMark-3.0 | 1,000 | 25% | 32.70 ±1.48 | 32.90 ±1.48 |