GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
raihan-js
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
raihan-js
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Parameters | ~27.7M (10M body, rest in embeddings) |
| Architecture | GPT-2 (GPT2LMHeadModel) |
| Training | From scratch — no base model |
| Vocabulary | 5,000 (custom medical tokenizer) |
| Context length | 512 tokens |
| Domain | Medical / clinical / biomedical English |
| Hardware | NVIDIA RTX 3060 12GB |
| License | Apache 2.0 |
Large medical LLMs (PubMedBERT, BioGPT, Med-PaLM) are expensive to run and require infrastructure most clinics, students, and researchers don't have. MedLLM-10M is a deliberately tiny, from-scratch GPT-2 trained on medical text — small enough to run on a CPU, finetune on a laptop, or use as a teaching artifact in an NLP course.
It is not competitive with commercial medical AI for diagnostic accuracy. It is competitive at being a small, transparent, fully open baseline for the medical domain.
Standard GPT-2 with small dimensions:
text
Architecture: GPT2LMHeadModelLayers: 8Hidden size (n_embd): 512Attention heads: 8Feed-forward (n_inner):2,048Max position (n_ctx): 512Activation: GELUVocab size: 5,000Dropout: 0.1| Epochs | 10 |
| Batch size | 4 |
| Gradient accumulation | 8 |
| Learning rate | 3e-4 |
| Optimizer | AdamW |
| Weight decay | 0.01 |
| Warmup steps | 200 |
| Mixed precision | FP16 |
| Sequence stride | 256 |
No pretrained weights. No transfer learning. The model is initialized fresh and trained end-to-end on medical text.
Loads with standard transformers:
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "raihan-js/medllm-10m"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id) prompt = "The patient presents with chest pain and"inputs = tokenizer(prompt, return_tensors="pt")output = model.generate(**inputs, max_new_tokens=120, temperature=0.7, do_sample=True)print(tokenizer.decode(output[0], skip_special_tokens=True))raihan-js/orch-fusion and the rest of the ORCH series — sibling from-scratch SLMs, in the code-generation domainAkteruzzaman Raihan Sikder — AI/ML engineer. Founding engineer and AI/ML lead at VETR Proposal; previously CTO of ClarioScope AI (2024–2026, sunset). Portfolio · GitHub.
bibtex
@misc{sikder2025medllm10m, title = {MedLLM-10M: A Lightweight GPT-2-Style Language Model Trained From Scratch on Medical Literature}, author = {Sikder, Akteruzzaman Raihan}, year = {2025}, url = {https://huggingface.co/raihan-js/medllm-10m}}Architecture: GPT2LMHeadModelLayers: 8Hidden size (n_embd): 512Attention heads: 8Feed-forward (n_inner):2,048Max position (n_ctx): 512Activation: GELUVocab size: 5,000Dropout: 0.1from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "raihan-js/medllm-10m"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id) prompt = "The patient presents with chest pain and"inputs = tokenizer(prompt, return_tensors="pt")output = model.generate(**inputs, max_new_tokens=120, temperature=0.7, do_sample=True)print(tokenizer.decode(output[0], skip_special_tokens=True))@misc{sikder2025medllm10m, title = {MedLLM-10M: A Lightweight GPT-2-Style Language Model Trained From Scratch on Medical Literature}, author = {Sikder, Akteruzzaman Raihan}, year = {2025}, url = {https://huggingface.co/raihan-js/medllm-10m}}