GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
DedeProGames
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
DedeProGames
Model Tree
Input Modalities
Output Modalities
Supported Functionality
This repository holds the weights at optimizer step 81,500 of a run targeting 200B tokens. Three things follow from that, and all three matter when reading the benchmark numbers:
| Field | Value |
|---|---|
| Tokens seen by this checkpoint | 42,729,472,000 (42.7B) |
| Target token budget | 200,000,000,000 (200B) |
| Progress | 21.4% |
| Optimizer step | 81,500 |
| Planned total steps | ~381,470 |
| Learning rate at this checkpoint | 1.2e-3 (peak, stable phase) |
| LR decay begins at step | ~343,320 (final 10% of the run) |
The learning rate has not decayed yet. The run uses a warmup-stable-decay schedule whose decay phase covers only the final 10% of steps. This checkpoint sits deep in the stable phase, still at the peak LR of 1.2e-3. Models evaluated mid-plateau consistently score below what the same weights reach after annealing, so the numbers below understate the model rather than describe it.
It has seen fewer tokens than Kiyo-135M. Kiyo-135M was trained on the full 200B tokens. This preview has seen 42.7B. A parameter-count comparison between the two is not a fair comparison of the architectures — it is a comparison of a finished run against a fifth of one.
Weights are not final and will change. Do not treat this checkpoint as the Kiyo-230M release. It exists so the run can be inspected mid-flight; the final model will be published separately.
| Field | Value |
|---|---|
| Parameters | 229,688,064 |
| Architecture | Llama-style decoder (SmolLM2 architecture family) |
| Layers | 32 |
| Hidden size | 768 |
| Intermediate size | 1,920 |
| Attention heads | 12 |
| KV heads | 4 |
| Head dimension | 64 |
| Attention type | Grouped query attention |
| Source | Domain | Mixture share |
|---|---|---|
| FineWeb-Edu | General web text, education-filtered | 45.0% |
| DCLM-Baseline | General web text, high-quality filtered | 30.0% |
| Stack-v3-train | Source code | 17.5% |
| FineMath | Mathematical reasoning | 7.5% |
The Stack-v3 portion is sampled with per-language keep probabilities across 32 curated programming languages, giving each language an equal token quota rather than inheriting the corpus's natural skew.
Self-reported result from the official BananaMind Base Bench 1.1 script, measured on CUDA in float32.
| Model | Params | Training tokens | Overall Elo |
|---|---|---|---|
| Kiyo-135M | 134.5M | 200B | 1,126 |
| BananaMind-2-Pro | 139.0M | 100B | 1,124 |
| Rose-Pro | 151.3M | — | 1,105 |
| Kiyo-230M-Preview | 229.7M | 42.7B |
Figures for BananaMind-2-Pro, Rose-Pro, and GPT-2 are as self-reported on their own model cards, all against the same BananaMind Base Bench 1.1 suite. This preview sits below Kiyo-135M despite having 1.7× the parameters, which is the expected result at 21% of the token budget with the learning rate still at peak — see Preview Status.
Detailed Kiyo-230M-Preview result
| Category | Accuracy | z vs. chance | Elo | Significant |
|---|---|---|---|---|
| Language completion | 98.0% | +11.92 | 1,468 | * |
| Code completion | 74.0% | +8.00 | 1,286 | * |
| World knowledge | 72.0% | +7.68 | 1,079 | * |
* = passes 1.96σ vs. chance; n=50 per category
Every category clears the significance threshold, including Quantitative — the one category where Kiyo-135M does not.
By difficulty
| Difficulty | Accuracy |
|---|---|
| Easy | 73.5% |
| Medium | 60.7% |
| Hard | 54.3% |
Summary
| Metric | Value |
|---|---|
| Parameters | 229,688,064 |
| Overall Elo | 1,086 |
| Chance floor | 805 |
| Above chance floor | +282 |
| Raw accuracy | 62.9% |
| 95% CI on accuracy | 57.8% – 67.9% |
Scores are self-evaluated and may vary with the benchmark revision, Transformers version, dtype, hardware, and generation settings. With 350 items, the 95% interval spans roughly ±5 accuracy points, so small Elo gaps between models should not be read as decisive.
bash
pip install -U transformers safetensors torch
python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLMmodel_id = "DedeProGames/Kiyo-230M-Preview"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id,torch_dtype=torch.bfloat16 if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else torch.float16,).cuda().eval()prompt = "The meaning of life is "input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(model.device)with torch.no_grad():output = model.generate(input_ids=input_ids,max_new_tokens=64,do_sample=False,repetition_penalty=1.1,pad_token_id=tokenizer.eos_token_id,eos_token_id=tokenizer.eos_token_id,)print(tokenizer.decode(output[0], skip_special_tokens=True))
This is an unfinished base model, not instruction-tuned — it continues text rather than following instructions, and its weights are an intermediate checkpoint rather than a release. Beyond the preview caveats above, it shares the limits of its scale: fluent and grammatical on language completion, competent on code, but weak on multi-step logic and context tracking, where it stays near chance on the harder items. The 2,048-token context window limits long-document use. It can generate incorrect facts and should not be used for high-stakes decisions without verification. Keep a finite generation limit to avoid repetition or drift on long outputs.
Apache 2.0
| Activation | SwiGLU |
| Normalization | RMSNorm |
| Positional encoding | RoPE (theta 10,000) |
| Vocabulary size | 49,152 |
| Context length | 2,048 |
| Embeddings | Tied input/output embeddings |
| Training tokens (this checkpoint) | 42,729,472,000 |
| Tokens per optimizer step | 524,288 |
| Peak learning rate | 1.2e-3 |
| LR schedule | Warmup-stable-decay (1/sqrt), decay not yet reached |
| Weight format | safetensors |
| GPT-2 | 124M | 10B | 990 |
| Commonsense |
| 70.0% |
| +7.35 |
| 1,077 |
| * |
| Context tracking | 44.0% | +3.10 | 931 | * |
| Logical reasoning | 44.0% | +3.10 | 1,017 | * |
| Quantitative | 38.0% | +2.12 | 958 | * |