GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
HuggingFaceTB
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
HuggingFaceTB
Model Tree
Input Modalities
Output Modalities
Supported Functionality
This model implements key DeepSeek-V4 innovations at a miniature scale:
| Component | Details |
|---|---|
| Parameters | ~110M total (41M embeddings, 69M non-embedding) |
| Hidden size | 320 |
| Layers | 8 |
| Attention heads | 8 (1 KV head — MQA-style) |
| Head dim | 96 (32 RoPE + 64 NoPE) |
| MLA | q_lora_rank=160, o_groups=2, o_lora_rank=80 |
| MoE | 4 routed experts + 1 shared, top-2 routing |
| Expert FFN | SwiGLU, intermediate_size=640 |
| Routing | sqrtsoftplus scoring, noaux_tc method |
| Hyper-Connections | hc_mult=4, Sinkhorn routing (2 iters) |
| MTP | 1 next-token prediction layer |
| Vocab | 129,280 (DeepSeek-V4 tokenizer) |
| Context | 2,048 tokens |
| Metric | Value |
|---|---|
| Final loss | ~5.3 (cross-entropy) |
| Final entropy | 3.77 |
| Token accuracy | 33.8% |
python
import torchfrom safetensors.torch import load_filefrom transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizerfrom huggingface_hub import hf_hub_download # Load model (recommended: manual load for reliability)config = AutoConfig.from_pretrained("HuggingFaceTB/nanowhale-100m-base", trust_remote_code=True)model = AutoModelForCausalLM.from_config(config, trust_remote_code=True).float() # Download and load weightsweights_path = hf_hub_download("HuggingFaceTB/nanowhale-100m-base", "model.safetensors")state_dict = load_file(weights_path)model.load_state_dict(state_dict, strict=True)model = model.cuda().eval() tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/nanowhale-100m-base") # Generateinput_ids = tokenizer.encode("The meaning of life is", return_tensors="pt").cuda()output = model.generate(input_ids, max_new_tokens=100, temperature=0.7, top_p=0.9, pad_token_id=tokenizer.eos_token_id)print(tokenizer.decode(output[0], skip_special_tokens=True))trust_remote_code=TrueApache-2.0
import torchfrom safetensors.torch import load_filefrom transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizerfrom huggingface_hub import hf_hub_download # Load model (recommended: manual load for reliability)config = AutoConfig.from_pretrained("HuggingFaceTB/nanowhale-100m-base", trust_remote_code=True)model = AutoModelForCausalLM.from_config(config, trust_remote_code=True).float() # Download and load weightsweights_path = hf_hub_download("HuggingFaceTB/nanowhale-100m-base", "model.safetensors")state_dict = load_file(weights_path)model.load_state_dict(state_dict, strict=True)model = model.cuda().eval() tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/nanowhale-100m-base") # Generateinput_ids = tokenizer.encode("The meaning of life is", return_tensors="pt").cuda()output = model.generate(input_ids, max_new_tokens=100, temperature=0.7, top_p=0.9, pad_token_id=tokenizer.eos_token_id)print(tokenizer.decode(output[0], skip_special_tokens=True))