AsifZaman1912

ChatgaiyyaLM-0.5B

README

License: apache-2.0

Quickstart

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

REPO = "AsifZaman1912/ChatgaiyyaLM-0.5B"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B", dtype=torch.float16)
model.resize_token_embeddings(len(tok))       # REQUIRED: adapter adds 3,914 tokens
model = PeftModel.from_pretrained(model, REPO).eval()

prompt = "চাটগাঁইয়া থেকে প্রমিত বাংলায় অনুবাদ করো:\nঅ্যাঁই ভাত ন হাইয়্যুম\nঅনুবাদ:"
ids = tok(prompt, return_tensors="pt")
out = model.generate(**ids, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Load the tokenizer from this repo. Token ids were assigned in insertion order, so a tokenizer rebuilt from elsewhere silently misaligns every added token.

Measured performance — read this carefully

chrF on the gold translation track of ChatgaiyyaBench, 120 fixed-seed items per direction.

Table with columns: System, ctg → bn, bn → ctg
System	ctg → bn	bn → ctg
base Qwen2.5-0.5B, no adaptation	20.54	19.77
copy the input unchanged	29.51	28.79
this adapter (independently re-measured)	35.66	24.10
rule transducer, no GPU	—	57.94
the same arm, as measured on the training run	58.46	52.63

The last two rows are the honest part.

The training run reported 58.46 / 52.63. Re-measuring these published weights locally gave 35.66 / 24.10. The weights here are packaged from a size-reduced archive of that run, and the round trip loses accuracy for a reason I was unable to isolate — I ruled out tokenizer misalignment (3,914 tokens, zero id mismatches), fp16 precision (fp32 scored identically), and a mis-targeted embedding freeze. The numbers in bold are what these files actually do.
A rule-based transducer with no GPU cost beats every neural arm at Bangla → Chittagonian. That is a real finding about the task, not an omission.

The adapter still clears both meaningful baselines — the unadapted model (20.54) and copying the input (29.51) — so it is a working dialect adapter, just not as strong as the training run suggested. A re-trained release without the lossy repackaging step is in progress.

Training

Table

Adaptation	vocabulary augmentation + subset-selected continued pretraining + instruction tuning
CPT tokens	7,189,504
CPT loss	2.7438
SFT loss	1.3713
Wall time	95 min on 2× Tesla T4
Added vocabulary	3,914 Chittagonian subword tokens
Precision	fp16+GradScaler (Tesla T4, sm_75 has no hardware bf16)

Vocabulary augmentation cuts Chittagonian fertility from 5.722 to 2.254 tokens/word. Across five tokenizers, the more Bangla-specialised the tokenizer, the larger its dialect penalty — banglat5 has the best absolute Bangla fertility (1.054) and the worst dialect tax (+72.6%).

Limitations

0.5B parameters. Qwen2.5-1.5B needs ~14.5 GB before activations under PEFT's duplicated embeddings and OOMs a 16 GB T4.
Fixed instruction-tuning budget, not full epochs.
Four of eight benchmark tracks are silver (auto-derived labels or transduced text), awaiting native-speaker validation.
Research artifact, not a production translator. For Bangla → Chittagonian specifically, the rule transducer is better.

Citation

bibtex
@software{chatgaiyyalm,
  title  = {ChatgaiyyaLM: Language Adaptation and Benchmarking for Chittagonian},
  author = {Asifuzzaman, A. K. M.},
  year   = {2026},
  url    = {https://github.com/A-K-M-Asifuzzaman/Chatgaiya-AI}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

AsifZaman1912

Model Tree

Base

Qwen/Qwen2.5-0.5B

Adapter

this model

Input Modalities

Text

Output Modalities