
KrynexAI 1.5
KrynexAI 1.5 is the model that refers to the KrynexAI Main family. It has 250M params, 40B tokens trained on, with LLaMa arch. It is a small base model that is probably going to generate incorrect or biased text, and it has neither been aligned nor evaluated for safety.

⚙️ Config
Table with columns: Spec, Details| Spec | Details |
|---|
| Parameters | 250M |
| Tokens seen | ~40.4B |
| Arch | Llama |
| Layers | 32 |
| Hidden size | 768 |
| Intermediate size (MLP) | 1920 |
| Attention heads | 12 (head dim 64) |
| KV heads (GQA) | 4 |
| Vocabulary size | 49,152 |
| Context length | 2,048 |
| RoPE theta | 10,000 |
| Activation | SiLU (SwiGLU MLP) |
| Tied embeddings | Yes |
| Precision | bfloat16 |
💻 Hardware
This model was trained on a single A100 SXM.
Benchmarks
Table with columns: Benchmark, Shots, acc, acc_norm, Random| Benchmark | Shots | acc | acc_norm | Random |
|---|
| HellaSwag | 0 | 31.64 | 37.18 | 25.0 |
| PIQA | 0 | 65.78 | 64.91 | 50.0 |
| ARC-Easy | 0 | 57.11 | 50.46 | 25.0 |
| ARC-Challenge | 0 | 25.00 | 26.28 | 25.0 |
| Arithmark-v3 | 0 | 38.00 | 38.20 | - |
| Open SLM leaderboard's II | - | 20.64 | - | - |
🚀 How to use
This code should download the model and generate a response.
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLMcheckpoint = "KrynexLabs/KrynexAI-1.5-250M-Instruct-Mobile"device = "cuda" if torch.cuda.is_available() else "cpu"tokenizer = AutoTokenizer.from_pretrained(checkpoint)model = AutoModelForCausalLM.from_pretrained( checkpoint, torch_dtype=torch.bfloat16, device_map="auto")inputs = tokenizer.encode("Gravity is", return_tensors="pt").to(device)outputs = model.generate(inputs, max_new_tokens=50)print(tokenizer.decode(outputs[0], skip_special_tokens=True))
📜 License
This model is released under Apache-2.0 License — you can use, modify, and share this code freely, even in commercial projects. The only requirement is to keep the original attribution. Full text in the file: LICENSE.