GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
k2walia
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
k2walia
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Architecture | Llama (decoder-only) |
| Parameters | 125,848,320 (~125.8M) |
| Layers / hidden / heads | 12 / 768 / 12 |
| Context length | 1024 |
| Vocab | 16,384 (byte-level BPE, trained on this corpus) |
| Tied embeddings | yes |
| Tokens seen | 1.76B (1 epoch) |
| Optimizer steps | 3,360 |
| Hardware | 8× H100 (data-parallel) |
| Wall-clock | ~12 minutes |
| Compute cost | ~$6–7 |
| Validation perplexity | 27.26 |
Optimizer AdamW (betas 0.9/0.95, wd 0.1), cosine LR 6e-4→6e-5 with 200M-token warmup, global batch 524,288 tokens, seq len 1024, bf16.
Built from public, ungated sources, streamed and cleaned from scratch:
| Source | Share | What |
|---|---|---|
| MedRAG/pubmed | ~82% | PubMed title+abstract snippets |
| HuggingFaceFW/fineweb-edu | ~12% | educational web (fluency) |
| epfl-llm/guidelines | ~5% | clinical practice guidelines |
| MedRAG/textbooks | ~1% | USMLE medical textbooks |
The corpus was decontaminated against PubMedQA, MedMCQA, and MedQA-USMLE (13-gram overlap removed) so downstream medical-QA evaluation is not leaked.
Base model for research and as a starting point for fine-tuning (e.g. medical QA). It is not a medical device and must not be used for clinical decisions or to provide medical advice. Outputs may be inaccurate, outdated, or fabricated.
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("k2walia/slm-125m-med-base")model = AutoModelForCausalLM.from_pretrained("k2walia/slm-125m-med-base")ids = tok("The patient presented with acute onset dyspnea and", return_tensors="pt")out = model.generate(**ids, max_new_tokens=64, temperature=0.8, do_sample=True)print(tok.decode(out[0], skip_special_tokens=True))