oddadmix

Emhotob-10M-Darija-English-v2

README

License: apache-2.0

Evaluation

Deterministic held-out set of 3,000 pairs (seed=42), decoded greedily (do_sample=False, no repetition penalty), scored with sacreBLEU:

Table with columns: Direction, sacreBLEU, chrF
Direction	sacreBLEU	chrF
Darija → English	19.84	29.42
English → Darija	21.96	27.77

Saved weights are the best checkpoint by validation loss (eval_loss = 2.093). 20 samples per direction with references are in eval_bidirectional.json.

Example translations

Real greedy-decoded outputs from the held-out set:

Darija → English

Table with columns: Source, Model output, Reference
Source	Model output	Reference
لا، عندنا تذاكر يا حبيبة	No, we're going to the game	No, we have tickets darling
غطّا وجهو و بكا	he is the first time	He covered his face and wept

English → Darija

Table with columns: Source, Model output, Reference
Source	Model output	Reference
No, we have tickets darling	لا، عندنا شي حاجة خايبة	لا، عندنا تذاكر يا حبيبة
He covered his face and wept	kan7al l7al l7al l7al	غطّا وجهو و بكا

Usage

ChatML format. Pick the system prompt for the direction you want:

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "oddadmix/Emhotob-10M-Darija-English-v2"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).to("cuda").eval()

SYSTEM = "You are a professional translator. Translate the Moroccan Darija text into English."

def translate(text, system=SYSTEM):
    prompt = (f"<|im_start|>system\n{system}<|im_end|>\n"
              f"<|im_start|>user\n{text.strip()}<|im_end|>\n<|im_start|>assistant\n")
    ids = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
    if tok.bos_token_id is not None:
        bos = torch.tensor([[tok.bos_token_id]], device=model.device)
        ids["input_ids"] = torch.cat([bos, ids["input_ids"]], dim=1)
        ids["attention_mask"] = torch.cat([torch.ones_like(bos), ids["attention_mask"]], dim=1)
    out = model.generate(**ids, max_new_tokens=256, do_sample=False,
                         eos_token_id=tok.eos_token_id, pad_token_id=tok.pad_token_id)
    return tok.decode(out[0, ids["input_ids"].size(1):], skip_special_tokens=True).strip()

Training

Base model: oddadmix/Emhotob-10M-v2 (Llama arch, hidden 256, 4 layers, 8 heads, vocab 32000, tied embeddings; 10,947,328 params after resizing for 2 ChatML tokens)
Dataset: oddadmix/darija_english_msa_parallel_dataset
Method: HuggingFace Trainer, ChatML, prompt-masked cross-entropy (loss only on the assistant turn). Each row is exploded into two training examples (one per direction).
Hyperparameters: 3 epochs · effective batch 64 · LR 3e-4 (cosine, 5% warmup) · bf16 · max length 1024 · load_best_model_at_end on eval_loss.
Eval split: 3,000 deterministic held-out pairs (seed=42), scored both directions.

Limitations

A ~10.9M model: reliable on short/common sentences, but drift, repetition, and errors appear on long or rare inputs. Gender is disambiguated only from context. For fluent translation use the 50M sibling.

License

Apache-2.0, inherited from the base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

oddadmix

Model Tree

Base

oddadmix/Emhotob-10M-v2

Fine-tuned

this model

Input Modalities

Text

Output Modalities