GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AnandHaridas1980
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
AnandHaridas1980
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Parameters | 125,847,552 (125.8M) |
| Layers / hidden / heads | 12 / 768 / 12 (MHA) |
| Context length | 1024 |
| Vocab | 16,384 (byte-level BPE, trained on this corpus) |
| Position encoding | RoPE (theta 10000) |
| Activation / norm | SwiGLU / RMSNorm |
| Tied embeddings | yes |
| Source | Tokens | Share |
|---|---|---|
| case-law | 716M | 35.1% |
| sec | 860M | 42.2% |
| fineweb-edu | 465M | 22.8% |
Built from HFforLegal/case-law (US court opinions), PleIAs/SEC (SEC filings) and
HuggingFaceFW/fineweb-edu (sample-10BT, general fluency filler). The legal sources
are the binding constraint: together they hold only ~2B clean tokens, so the mix is
"take all the legal text, add a small web slice" rather than a chosen ratio.
Pipeline: stream -> 6-step deterministic clean (line filters, boilerplate strip, 4-gram repetition, ASCII/langdetect English gate, dictionary-based OCR gate on case-law) -> MinHash near-dedup + exact dedup -> 13-gram decontamination against CaseHOLD/LexGLUE -> pack into 1024-token windows, 99/1 train/val split.
| Tokens seen | 8.16B (4 epochs) |
| Steps | 15,568 |
| Global batch | 524,288 tokens |
| Optimizer | AdamW (betas 0.9/0.95, wd 0.1, clip 1.0) |
| LR schedule | cosine 0.0006 -> 6e-05, 200M warmup tokens |
| Precision | bf16 autocast, fp32 master weights |
| Hardware | 8x NVIDIA H100 (DDP, single node) |
| Final val loss | 2.1228 |
| Split | Perplexity |
|---|---|
| ALL | 8.31 |
| case-law | 8.68 |
| sec | 4.80 |
| fineweb-edu | 21.61 |
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("AnandHaridas1980/slm125m-live")model = AutoModelForCausalLM.from_pretrained("AnandHaridas1980/slm125m-live")ids = tok("The plaintiff filed a motion to dismiss on the grounds that", return_tensors="pt")print(tok.decode(model.generate(**ids, max_new_tokens=60, do_sample=True, top_p=0.95)[0]))
Base model, English only, 1024-token context. The case-law source is OCR'd and retains some scanning noise despite the dictionary gate. Training data is skewed to older SEC filings. Do not use for legal or financial advice.