GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
japawblob
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
japawblob
Model Tree
sbintuitions/sarashina2-70b
Input Modalities
Output Modalities
Supported Functionality
Sarashina2-70b uses rotary position embeddings (RoPE). RoPE has no learned
position parameters — the cos/sin tables are computed analytically from the
position index at runtime — so there is no position-embedding matrix to resize.
The 8192 ceiling existed only in config.json.
The weights are byte-for-byte identical to the base model. The complete diff:
diff
# config.json- "max_position_embeddings": 8192,+ "max_position_embeddings": 32768,# tokenizer_config.json (so the tokenizer doesn't truncate first)+ "model_max_length": 32768,
rope_theta stays at 10000.0 and rope_scaling stays null, so the attention
math is identical to the base model — timing differences are attributable to
sequence length alone. A companion variant applying linear position
interpolation ({"rope_type": "linear", "factor": 4.0}) is a strictly better
choice if you care at all about output degrading gracefully.
32768 was chosen over a bare 24576 to leave headroom for a 24k prompt plus generation.
| Params | 70.13 B |
| Layers | 80 |
| Hidden / FFN | 8192 / 28672 |
| Attention heads | 64 (8 KV heads, GQA) |
| Head dim | 128 |
| Vocab | 102400 |
| dtype | bfloat16 |
| Context | 32768 (was 8192) |
A single 24k-token sequence needs ~148 GB total. Practical minimum is
2x H100/H200 80GB (--tensor-parallel-size 2), or FP8/AWQ quantisation to
~70 GB.
bash
vllm serve <this-repo> \--max-model-len 24576 \--tensor-parallel-size 2 \--dtype bfloat16 \--gpu-memory-utilization 0.92
[!TIP] You do not actually need this repo. The same effect is achievable against the original model without downloading anything extra:
bash
vllm serve sbintuitions/sarashina2-70b --max-model-len 24576 \--hf-overrides '{"max_position_embeddings": 32768}'python
AutoModelForCausalLM.from_pretrained("sbintuitions/sarashina2-70b",max_position_embeddings=32768)This repo just packages that as a fixed, reproducible artifact.
model.safetensors.index.json total_size
exactly; 723/723 tensors parsed; 70.13 B params; all BF16LlamaDecoderLayer.
Finite output over (1, 24576, 8192); activation norms stable past the old
ceiling (pos 8191 → 2.71, pos 24575 → 2.78), i.e. no numerical blow-up.Not verified: quality/perplexity at >8k, and stacking all 80 layers (no GPU was available). Assume quality is bad.
MIT, inherited from the base model. Copyright (c) 2024 SB Intuitions — the
original LICENSE is retained unmodified in this repo. All model weights are
the work of SB Intuitions; the only contribution here is a two-line config edit.