GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
jonam-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
jonam-ai
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Architecture | Llama (via transformers.LlamaForCausalLM) |
| Parameters | 125.8M (tied embeddings) |
| Vocab | 16,384 (byte-level BPE trained on this corpus) |
| Layers / hidden / heads | 12 / 768 / 12 (head dim 64, MHA) |
| Context length | 1,024 |
| Positional | RoPE (θ=10,000) |
| Norm / activation | RMSNorm (1e-5) / SwiGLU (silu) |
| Precision | bf16 training, weights saved fp32 |
Streamed, cleaned, deduplicated (MinHash LSH + exact), and decontaminated against CaseHOLD / LexGLUE before tokenization. Realized token mix:
| Source | Share | HF dataset |
|---|---|---|
| US case law | ~35% | HFforLegal/case-law (split us) |
| SEC filings | ~42% | PleIAs/SEC |
| Educational web | ~23% | HuggingFaceFW/fineweb-edu (sample-10BT) |
torch.compile, SDPA/flash attention)Held-out validation perplexity: 9.13 (loss 2.211, full 1% held-out split, 20,581,737 tokens / 20,119 packed 1024-token windows).
Validation loss over training (subset eval): 2.796 → 2.232 (steps 1000 → 7778).
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torchtok = AutoTokenizer.from_pretrained("s2211252/slm-125m-base")model = AutoModelForCausalLM.from_pretrained("s2211252/slm-125m-base", torch_dtype=torch.bfloat16)prompt = "Pursuant to the terms of this Agreement, the parties"ids = tok(prompt, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=120, do_sample=True, top_k=50, top_p=0.95, temperature=0.8)print(tok.decode(out[0], skip_special_tokens=True))
Small base model, English only, 1,024-token context. Trained on legal/financial + web text; it is not instruction-tuned and can produce inaccurate or fabricated legal/financial statements. Not for legal advice or production decisions without review. Domain contamination against CaseHOLD/LexGLUE was filtered, but standard LM caveats apply.