Overview
This repository contains the NVFP4-quantized version of RastNus-Small-5B for fast serving with vLLM.
RastNus (ڕاستنووس, "correct writing") is a 5-billion-parameter (~2B effective) language model specialized in grammar, spelling, and punctuation correction for Central Kurdish (Sorani / کوردیی ناوەندی).
Kurdish is one of the most under-served languages in modern NLP. Everyday written Sorani commonly contains orthographic inconsistencies — missing diacritic letters (ی/ێ, و/ۆ, ر/ڕ), spacing errors, informal punctuation, and mixed loanword usage. RastNus takes raw Sorani text and returns a clean, standardized, correctly written version, while preserving the author's meaning and voice.
RastNus is developed and released by PawanKrd as part of our commitment to bringing state-of-the-art AI to the Kurdish language and its speakers.
Key Capabilities
- Spelling correction — fixes common orthographic errors and non-standard letter usage (e.g.
کتیب → کتێب, زور → زۆر, مروف → مرۆڤ)
- Grammar correction — repairs agreement, morphology, and word-form errors
- Punctuation control — optionally adds and normalizes punctuation, or leaves punctuation untouched, via a dedicated control token
- Pure Kurdish mode — optionally replaces non-Kurdish loanwords with pure Kurdish equivalents, via a dedicated control token
- Meaning-preserving — corrects without paraphrasing, summarizing, or adding content
Model Family
Table with columns: Model, Params, bf16, FP8, NVFP4, GGUF| Model | Params | bf16 | FP8 | NVFP4 | GGUF |
|---|
| RastNus-Large | – | Not publicly available - Try it now | – | – | – |
| RastNus-Medium | – | Under development | – | – | – |
| RastNus-Base | 12B | bf16 | FP8 | NVFP4 | GGUF |
| RastNus-Small | 5.1B | bf16 | FP8 | ✅ You are here | GGUF |
| RastNus-Tiny | 1.03B | bf16 | FP8 | NVFP4 | GGUF |
Scores for every model and format are in the Benchmarks section below.
Benchmarks
Every RastNus model and format was evaluated on the same internal held-out test set of 329 Central Kurdish correction examples, none of which were seen during training by any RastNus model, using greedy decoding. CER / WER = character / word error rate against the reference correction (lower is better). For reference, returning the input unchanged scores 6.34% CER. Differences of about ±0.2% CER are within noise.
vLLM — bf16, FP8, NVFP4
Table with columns: Model, Format, Size, CER ↓, WER ↓, Exact match ↑, Throughput ↑| Model | Format | Size | CER ↓ | WER ↓ | Exact match ↑ | Throughput ↑ |
|---|
| RastNus-Base-12B | bf16 | 23.9 GB | 3.68% | 11.63% | 20.7% | 653 tok/s |
| RastNus-Base-12B-FP8 | FP8 | 13.1 GB | 3.75% | 11.64% | 21.3% | 1,009 tok/s |
| RastNus-Base-12B-NVFP4 | NVFP4 | 11.7 GB | 3.54% | 11.80% | 17.3% | 1,071 tok/s |
| RastNus-Small-5B | bf16 | 10.2 GB | 3.67% | 10.76% | 17.6% | 2,783 tok/s |
| RastNus-Small-5B-FP8 | FP8 | 8.4 GB | 3.62% | 10.73% | 17.6% | 3,521 tok/s |
| RastNus-Small-5B-NVFP4 | NVFP4 | 8.0 GB | 3.72% | 11.29% | 17.9% | 3,783 tok/s |
| RastNus-Tiny-1B | bf16 | 2.1 GB | 4.77% | 13.08% | 19.5% | 4,777 tok/s |
| RastNus-Tiny-1B-FP8 | FP8 | 1.6 GB | 4.81% | 13.00% | 19.5% | 5,148 tok/s |
| RastNus-Tiny-1B-NVFP4 | NVFP4 | 1.5 GB | 4.56% | 12.67% | 18.5% | 5,891 tok/s |
Throughput = generated tokens per second for the whole test set as a batch on a single NVIDIA RTX PRO 5000 Blackwell (vLLM 0.33).
llama.cpp — GGUF
Measured with llama.cpp (CUDA) on the same GPU. Same output as BF16 = share of examples where the quantized file produces exactly the same text as the lossless BF16 GGUF — the most direct measure of quantization loss. ⚠️ = not recommended (no better than leaving the text uncorrected). ¹ = one test input containing many blank lines makes the model loop on newlines until the length limit; without that single example these files score at the BF16 level (≈3.7% CER). Small differences between the vLLM and llama.cpp bf16 scores come from numerical differences between the two engines, not from the weights.
Example
Input
خویندنەوەی کتیب زور گرنگە بو مروف ، چونکە بیروهۆش دەکاتەوە ئەو کەسە ی کتێب دەخوێنەوە؛ ئاستی رۆشنبیری یان بەرز ترە لەوانی تر
Output
خوێندنەوەی کتێب زۆر گرنگە بۆ مرۆڤ، چونکە بیروهۆش دەکاتەوە. ئەو کەسەی کتێب دەخوێنێتەوە، ئاستی ڕۆشنبیرییان بەرزترە لەوانی تر.
Model Details
Table | |
|---|
| Developed by | PawanKrd |
| Model type | Causal decoder-only transformer (Gemma 4 architecture) |
| Base model | Google Gemma 4 E2B |
| Parameters | ~5.1B total (~2B effective; includes per-layer embeddings) |
| Language | Central Kurdish / Sorani (ckb), Arabic script |
| Task | Text correction (grammar, spelling, punctuation) |
| Precision | NVFP4 (MLP), bf16 (attention, embeddings) |
| Context length | Long-context (inherited from Gemma 4); recommended working length up to ~8K tokens per request |
| License | Apache 2.0 |
RastNus uses a task-specific prompt format with dedicated special tokens (added to the tokenizer vocabulary). The behavior of the model is controlled by two switches:
Table with columns: Token, Effect| Token | Effect |
|---|
<|punct:on|> / <|punct:off|> | Add & normalize punctuation, or leave punctuation untouched |
<|pure:on|> / <|pure:off|> | Replace non-Kurdish loanwords with pure Kurdish equivalents, or keep original vocabulary |
The full sequence layout:
<bos><|task:kurdish_correction|><|punct:on|><|pure:off|><|input|>{your Kurdish text}<|output|>{corrected text}<eos>
You do not need to build this manually — the repository ships with a chat template that handles it (see below). Defaults are punctuationEnabled=True and kurdishPureMode=False.
Quantization
NVFP4 (4-bit floating point) weights and activations for the MLP layers, produced with llm-compressor and calibrated on 512 in-domain Kurdish correction examples. Attention layers, embeddings (including the large per-layer embedding tables), lm_head and the vision/audio towers are kept in bf16.
Hardware: NVIDIA Blackwell GPUs (e.g. RTX 50xx, RTX PRO 6000/5000, B200).
See Benchmarks for the quality and speed of every format.
Usage
Requires pip install compressed-tensors (the quantized weights are loaded automatically).
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "PawanKrd/RastNus-Small-5B-NVFP4" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto") text = "خویندنەوەی کتیب زور گرنگە بو مروف" prompt = tokenizer.apply_chat_template( [{"role": "user", "content": text}], tokenize=False, add_generation_prompt=True, punctuationEnabled=True, # add/normalize punctuation kurdishPureMode=False, # keep loanwords as-is) inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Note: pass add_special_tokens=False when tokenizing — the chat template already emits <bos>.
vLLM (OpenAI-compatible server)
vllm serve PawanKrd/RastNus-Small-5B-NVFP4
from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="none") resp = client.chat.completions.create( model="PawanKrd/RastNus-Small-5B-NVFP4", messages=[{"role": "user", "content": "خویندنەوەی کتیب زور گرنگە بو مروف"}], extra_body={"chat_template_kwargs": { "punctuationEnabled": True, "kurdishPureMode": True, # enable pure-Kurdish word replacement }}, temperature=0.0,)print(resp.choices[0].message.content)
Recommended Generation Settings
- Greedy decoding (
do_sample=False / temperature=0) — correction is a deterministic task
- These defaults are built in:
generation_config.json sets greedy decoding, so model.generate() needs no extra sampling arguments
max_new_tokens ≈ 1.5 × input token count (the corrected output is roughly the same length as the input)
- For documents longer than ~4,000–5,000 characters, split at sentence/paragraph boundaries, correct each chunk, and rejoin
Intended Use & Limitations
Intended uses: proofreading and standardization of Sorani text in publishing, education, journalism, document processing, writing assistants, and any pipeline that needs clean, orthographically consistent Kurdish.
Limitations:
- The model targets Central Kurdish (Sorani) written in Arabic script. It is not designed for Kurmanji (Latin script), Zazaki, or other Kurdish varieties.
- It is a correction model, not a general chat assistant — it expects the task prompt format above and will not follow open-ended instructions.
- Like all language models, it can occasionally miss errors or over-correct rare proper nouns, dialect-specific spellings, or intentional stylistic choices. Human review is recommended for high-stakes publishing.
- Pure-Kurdish mode applies vocabulary substitutions that may not match every publication's style guide; review its output for terminology-sensitive content.
About PawanKrd
Pawan.Krd (PKRD, LLC) builds AI and cloud infrastructure — LLMs, ASR, TTS, and related APIs — with a focus on reliability and open models. RastNus is part of that work: first-class writing tools for Kurdish, released openly.
License & Attribution
This model is released under the Apache License, Version 2.0. See the LICENSE file for the full text and the NOTICE file for attribution details.
RastNus-Small-5B-NVFP4 is a quantized version of RastNus-Small-5B, a derivative work of Gemma 4 E2B by Google LLC / Google DeepMind, which is also licensed under Apache 2.0. In accordance with the license:
- The original Apache 2.0 license text is included in this repository.
- The
NOTICE file carries the required attribution to Google and a prominent statement that the model weights, tokenizer, and chat template have been modified by PawanKrd.
- Gemma is a trademark of Google LLC, used here only to describe the origin of the base model. This project is not affiliated with, endorsed by, or sponsored by Google.
This model is provided "AS IS", without warranties or conditions of any kind. See Sections 7 and 8 of the Apache License for the disclaimer of warranty and limitation of liability.
Citation
@misc{rastnussmall2026, title = {RastNus-Small-5B: A Central Kurdish (Sorani) Grammar and Spelling Correction Model}, author = {PawanKrd}, year = {2026}, url = {https://huggingface.co/PawanKrd/RastNus-Small-5B}, note = {Derivative of Google Gemma 4 E2B, Apache License 2.0}}