Why this model??
Most public language models start from billions of pretrained weights.
This one doesn't.
Every one of its 26.11 million parameters began as random numbers and learned exclusively from a curated Gen Z → English translation dataset.
Built specifically for:
- ⚡ Local inference
- 🧠 Slang translation
- 💻 GGUF & llama.cpp
- 🦙 Ollama
- 🤗 Transformers.
✨ See it in action
Table with columns: Gen Z, Standard English| Gen Z | Standard English |
|---|
no cap | I'm being completely honest. |
say less, I'm tryna vibe | I understand, I'm trying to relax. |
she's mogging everyone | She's outshining everyone by looking better. |
that's straight cap | That's an outright lie. |
after the argument she crashed out | After the argument she had an emotional outburst. |
The goal isn't to sound like ChatGPT.
The goal is to preserve meaning while removing slang naturally.
📊 Model Overview.
Table with columns: Property, Value| Property | Value |
|---|
| Architecture | Llama-style Decoder-only Transformer |
| Parameters | 26.07M |
| Layers | 7 |
| Hidden Size | 448 |
| Attention Heads | 7 |
| Context Length | 384 |
| Vocabulary | 8,000 |
| Tokenizer | Custom Byte-Level BPE |
| Training | From Scratch |
🎯 What it's good at
The model is intentionally specialized.
Excels at
- ✅ Gen Z slang
- ✅ TikTok vocabulary
- ✅ Internet abbreviations
- ✅ Meme language
- ✅ Social media captions
- ✅ Paragraph rewriting
Not designed for
- ❌ General chatting
- ❌ Coding
- ❌ Mathematics
- ❌ Knowledge retrieval
- ❌ Long conversations
Think of it as a translator, not a general assistant.
🏗️ Built from Scratch
This is the project's biggest differentiator.
Instead of fine-tuning Llama, Qwen, or Mistral, the entire network was trained from random initialization.
That means:
- Custom tokenizer
- Custom vocabulary
- No inherited knowledge
- Every weight learned only from the translation dataset
Architecture
Layers 7
Hidden Size 448
Attention Heads 7
KV Heads 7
Intermediate Size 1792
Context Length 384
Vocabulary 8000
Activation SiLU
RoPE ✓
RMSNorm ✓
Tied Embeddings ✓
Although it follows the Llama architecture, it does not reuse Meta's pretrained weights.
📚 Training Data
The model learned from 139,074 cleaned instruction-response pairs.
Split
Table with columns: Split, Size| Split | Size |
|---|
| Train | 90% |
| Validation | 5% |
| Test | 5% |
The split was deterministic using a fixed seed and stratified by instruction type.
Two prompt styles were used consistently:
- Single-sentence translation
- Paragraph translation
Example training prompt:
<s><|instruction|>Translate the following Gen Z slang sentence into clear, standard English.<|input|>bro that fit is so mid ngl<|response|>
Keeping one prompt format helped the model specialize instead of behaving like a chatbot.
🔤 Custom Tokenizer
The tokenizer wasn't borrowed either.
It was trained exclusively on the Gen Z dataset.
Table with columns: Property, Value| Property | Value |
|---|
| Type | Byte-Level BPE |
| Vocabulary | 8,000 |
| Training Data | Only this dataset |
Because of this custom tokenizer, additional compatibility work was required for GGUF conversion and llama.cpp support.
Internal evaluation on 50 randomly sampled test examples showed:
Table with columns: Metric, Score| Metric | Score |
|---|
| Exact Match | 69% |
| Semantic Match | ~97% |
| Crash Stability | 100% |
The exact-match metric is intentionally strict.
Many "incorrect" outputs are actually valid paraphrases.
Example:
Input
say less
Output
I understand.
Different wording.
Same meaning.
📦 Available Files
Table with columns: File, Purpose| File | Purpose |
|---|
model.safetensors | Hugging Face model |
config.json | Model configuration |
tokenizer.json | Custom tokenizer |
genz-translator-f16.gguf | Full precision |
genz-translator-q8_0.gguf | Recommended GGUF |
Recommended download
genz-translator-q8_0.gguf
Best balance between quality and local CPU performance.
⚡ Quick Start
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
repo = "Sankar-2910/genz-translator"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
prompt = (
"<s><|instruction|>"
"Translate the following Gen Z slang sentence into clear, standard English."
"<|input|>no cap"
"<|response|>"
)
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
output = model.generate(
**inputs,
max_new_tokens=64,
do_sample=False,
temperature=0,
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.eos_token_id
)
print(tokenizer.decode(
output[0][inputs.input_ids.shape[1]:],
skip_special_tokens=True
))
Output:
I'm being completely honest.
🦙 llama.cpp
llama-cli \
-m genz-translator-q8_0.gguf \
-p "<s><|instruction|>Translate the following Gen Z slang sentence into clear, standard English.<|input|>bro that fit is so mid ngl<|response|>" \
-n 96
⚙️ Ollama
Create a Modelfile.
FROM ./genz-translator-q8_0.gguf
TEMPLATE """<s><|instruction|>Translate the following Gen Z slang sentence into clear, standard English.<|input|>{{ .Prompt }}<|response|>"""
PARAMETER temperature 0
PARAMETER num_predict 160
PARAMETER stop "</s>"
Build:
ollama create genz-translator -f Modelfile
Run:
ollama run --raw genz-translator "bro that fit is so mid ngl"
Using --raw preserves the original training prompt format.
Use the same structure the model was trained on.
<s><|instruction|>Translate the following Gen Z slang sentence into clear, standard English.<|input|>no cap<|response|>
Avoid chat-style prompts.
The model is optimized for single-turn translation, not conversational memory.
💻 Hardware
This model was designed for lightweight local inference.
Table with columns: Quantization, Approx. Memory| Quantization | Approx. Memory |
|---|
| Q8_0 | ~30 MB |
| F16 | ~50 MB |
Recommended settings:
- 4–8 CPU threads
temperature=0
num_predict≈160
⚠️ Limitations
Being trained from scratch on a specialized dataset means:
- rare slang may vary by context,
- ambiguous abbreviations (like OP) depend on surrounding text,
- long generations can truncate if token limits are too low,
- general world knowledge is intentionally limited.
The trade-off is specialization: small, fast, and purpose-built.
📖 Citation
If you use this model in research, please cite:
@misc{genztranslator2026,
title={GenZ Translator},
author={Sankar Narayanan},
year={2026},
note={26M parameter decoder-only transformer trained entirely from scratch for Gen Z slang translation},
url={https://huggingface.co/Sankar-2910/genz-translator}
}
If referencing the underlying architecture:
@article{touvron2023llama,
title={LLaMA: Open and Efficient Foundation Language Models},
author={Touvron, Hugo and others},
journal={arXiv preprint arXiv:2302.13971},
year={2023}
}
❤️ Built by a Student
.
This project was created by Sankar Narayanan as an exploration of training compact language models from scratch, custom tokenization, GGUF deployment, and local AI inference.
If you build something with it, I'd love to see it.