GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
BerkayRA
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
BerkayRA
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Component | Value |
|---|---|
| Layers | 12 |
| Attention heads | 12 (head dim 64, no GQA) |
| Hidden size (d_model) | 768 |
| FFN | SwiGLU, intermediate 2048 |
| Context length | 1024 tokens |
| Positional encoding | RoPE (θ = 10,000) |
| Normalization | RMSNorm (pre-norm, ε = 1e-5) |
| Biases | none |
| Embeddings | tied input/output |
| Vocab | 32,000 |
| Parameters | ≈110 M total (~85 M non-embedding + 24.6 M tied token embedding); the project labels this config "124M" / "Config A" |
The architecture is standard Llama, so it loads directly with
AutoModelForCausalLM and is servable by TGI / vLLM / text-generation-inference.
SentencePiece unigram, vocab 32,000 (sp_unigram_32000).
Special tokens: unk=0, bos=1 (<s>), eos=2 (</s>), pad=3 (<pad>).
eos (2) doubles as the document separator in the training stream.
eos
between documents; ~39.3 B tokens seen over training.| Metric | Value |
|---|---|
| Train loss (step 300k) | 2.832 |
| Validation loss | 3.048 |
| Start loss (step 0) | 10.52 |
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerrepo = "BerkayRA/egemen-turkish-124m" # <- your repo idtok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.float16)prompt = "Türkiye'nin başkenti"# The tokenizer does not auto-prepend BOS; the model was trained with id 1 as BOS# and id 2 as the document separator / EOS.ids = torch.tensor([[1] + tok(prompt, add_special_tokens=False).input_ids])out = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.8,top_p=0.95, eos_token_id=2, repetition_penalty=1.3)print(tok.decode(out[0][1:], skip_special_tokens=True))
ONNX weights are included under onnx/: model.onnx (fp32) and
model_quantized.onnx (int8, ~135 MB) — so the model runs in the browser
via 🤗 transformers.js with no
server (see the live demo above), or under ONNX Runtime in Python.
js
import { AutoTokenizer, AutoModelForCausalLM, Tensor } from "@huggingface/transformers";const tok = await AutoTokenizer.from_pretrained("BerkayRA/egemen-turkish-124m");const model = await AutoModelForCausalLM.from_pretrained("BerkayRA/egemen-turkish-124m", { dtype: "q8" });const enc = await tok("Türkiye'nin başkenti", { add_special_tokens: false });const ids = [1n, ...Array.from(enc.input_ids.data, BigInt)]; // bos=1const input_ids = new Tensor("int64", BigInt64Array.from(ids), [1, ids.length]);const out = await model.generate({ input_ids, max_new_tokens: 60, do_sample: true,temperature: 0.8, top_p: 0.95, eos_token_id: 2,repetition_penalty: 1.3 });console.log(tok.decode(Array.from(out[0].data, Number).slice(ids.length), { skip_special_tokens: true }));
Intended: research on Turkish LM pre-training, tokenizer/architecture experiments, a small base model for further fine-tuning (SFT/LoRA), education.
Not intended: production use, factual question answering, or any safety-sensitive application. This is a base model — no instruction tuning, no RLHF, no safety alignment.
Limitations:
markdown
@misc{adanali2026turkishllm124m,title = {Egemen Turkish LM 124M (from scratch)},author = {Adanalı, Berkay},year = {2026},note = {From-scratch Llama-style Turkish base model, run u32\_124m}}