GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Convergent-ch
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Convergent-ch
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Trained for about 36 minutes on one RTX 3060 Ti at roughly 1 million tokens per second, about 2.5 billion tokens in total (around 220,000 tokens per parameter; the usual rule of thumb is 20).
| Training time (approx.) | Validation loss | Next-token accuracy |
|---|---|---|
| 0.5 min | 2.465 | 34.9 % |
| 3.5 min | 2.344 | 37.5 % |
| 7 min | 2.320 | 38.1 % |
| 14 min | 2.300 | 38.5 % |
| 36 min (end) | 2.269 | 39.0 % |
The model is practically full after about 5 minutes: the last 30 minutes improved the validation loss by only about 0.05. Its tokens are single bytes plus a few merges, so these numbers are not comparable with models that use larger vocabularies.
| Parameters | 11,088 |
| Architecture | Llama-style decoder (RMSNorm, SwiGLU, RoPE, tied embeddings) |
| Layers / hidden / MLP | 2 / 16 / 40 |
| Attention heads | 2 (head dim 8) |
| Context | 256 tokens |
| Vocabulary | 320 (byte-level BPE: 256 bytes, 8 special tokens, 56 merges) |
Download pico-1.3-small.gguf and Modelfile into one folder, then:
bash
ollama create pico1.3-small -f Modelfileollama run pico1.3-smallpython
from transformers import AutoTokenizer, AutoModelForCausalLM tok = AutoTokenizer.from_pretrained("Convergent-ch/Pico-1.3-Small")model = AutoModelForCausalLM.from_pretrained("Convergent-ch/Pico-1.3-Small") ids = tok.apply_chat_template([{"role": "user", "content": "Who made you?"}], add_generation_prompt=True, return_tensors="pt")out = model.generate(ids, max_new_tokens=40, do_sample=False)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))# I'm Pico 1.3 Small, Convergent.96,591 short chats (user message under 200 characters, reply under 240): first turns of conversations from TinyDialogues (MIT, generated with GPT-4 by its authors), Everyday Conversations and Smol-SmolTalk (Apache 2.0, generated with Llama 3.1 by their authors), plus 4,000 template chats about its name and maker. 150,000 steps of 64 x 256 tokens, AdamW, learning rate 5e-3 with cosine decay, bf16.
Apache 2.0.
ollama create pico1.3-small -f Modelfileollama run pico1.3-smallfrom transformers import AutoTokenizer, AutoModelForCausalLM tok = AutoTokenizer.from_pretrained("Convergent-ch/Pico-1.3-Small")model = AutoModelForCausalLM.from_pretrained("Convergent-ch/Pico-1.3-Small") ids = tok.apply_chat_template([{"role": "user", "content": "Who made you?"}], add_generation_prompt=True, return_tensors="pt")out = model.generate(ids, max_new_tokens=40, do_sample=False)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))# I'm Pico 1.3 Small, Convergent.