SatorTenet

gemma-4-E2B-it-serbian-dpo-lora

README

License: apache-2.0

What it does (and honestly does not)

Table with columns: capability, verdict
capability	verdict
Clean Serbian prose, both scripts, ekavian	✅ the main win — 0 script mixing, clean Markdown
Answering in the script you ask in	✅
Refusing false-premise questions	⚠️ partially — see the measured story below
Factual QA without retrieval	❌ it is a 2B model; use RAG for facts

Examples (greedy, actual outputs):

Q: Koje godine je Nikola Tesla dobio Nobelovu nagradu za fiziku? A: Nikola Tesla nije dobio Nobelovu nagradu za fiziku. …

Q: U kojoj srpskoj pokrajini se nalazi grad Dubrovnik? A: Grad Dubrovnik se ne nalazi u Srbiji. On se nalazi u Hrvatskoj…

The measured story

We tracked 20 false-premise trap questions (own SRZ-1 eval, frozen classifier + mandatory manual review) across every method. Confabulation count, lower is better:

Table with columns: step, traps confabulated (manual)
step	traps confabulated (manual)
base gemma-4-E2B-it	10/20 (it often abstained)
after SFT (form training)	20/20 — SFT destroys the base model's caution
SFT + 13 % diverse denial examples	~16–17/20 — imitation learning cannot restore it
+ DPO, 268 pairs	14/20
+ DPO, 465 pairs, 3 epochs (this adapter)	12/20

Three findings we believe transfer beyond this project:

SFT teaches form but destroys epistemic caution and cannot restore it. Abstention went 7/20 → 0/20 after any SFT; even 13 % diverse denial examples in the SFT mix left confabulation at ~17/20.
Fine-tuning does not inject retrievable facts into a 2B model. In a controlled run (3,000 facts, dose ladder, untrained control arm) the control arm moved most; trained arms stayed flat.
DPO is the only method that moved the confidence prior — with real generalization (new denials are on entities that were banned from the training set) — but it plateaus: +73 % pairs and +1 epoch bought ~2 traps; epochs 2 and 3 deny an identical trap set.

Knowledge guard: paired McNemar on MC accuracy after every step — final Δ latin −0.012 (p=0.42), cyrillic −0.004 (p=0.23) vs. the SFT checkpoint.

Usage

llama.cpp (recommended — 101 MB GGUF adapter included)

bash
llama-server -m gemma-4-E2B-it-Q6_K.gguf \
  --lora gguf/lima_dpo_adapter_v2_f16.gguf \
  -ngl 99 -c 131072 --jinja --reasoning off

--reasoning off is required. Without it the gemma-4 chat template puts the model in thinking mode and message.content comes back empty (everything lands in reasoning_content). Alternatively pass "chat_template_kwargs": {"enable_thinking": false} per request.

transformers + PEFT

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("google/gemma-4-E2B-it",
                                            device_map="auto", load_in_4bit=True)
model = PeftModel.from_pretrained(base, "SatorTenet/gemma-4-E2B-it-serbian-dpo-lora")
tok = AutoTokenizer.from_pretrained("SatorTenet/gemma-4-E2B-it-serbian-dpo-lora")

msgs = [{"role": "user", "content": "Napiši mi tri rečenice o Beogradu."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids.to(model.device), max_new_tokens=200)[0]))

Training details

Stage 1 — SFT (form): LIMA-style instruction set with masked-prompt loss, 2 epochs; fixed script mixing, repetition and formatting.
Stage 2 — DPO (calibration): 465 preference pairs — chosen = audited denials/abstentions, rejected = the SFT model's own greedy confabulations on the same premises; β=0.1, lr 5e-6, 3 epochs, manual DPO loop with reference log-probs precomputed. An entity guard (with Cyrillic transliteration) ensured zero overlap between training entities and eval traps — measured denials are generalization, not memorization.

Limitations

Factual knowledge is that of a 2B model: it still confabulates on 12/20 trap questions. Do not use it for factual QA without retrieval.
MC accuracy end-to-end: 0.478 latin / 0.406 cyrillic vs 0.514 / 0.432 base — a small cumulative tax from the SFT stage (guard-clean per step).
Occasional invented detail or ijekavian slip in free prose.
Own evaluation (SRZ-1, 20 traps) — small samples, wide intervals.

Files

Table with columns: file, size, purpose
file	size	purpose
`adapter_model.safetensors` + `adapter_config.json`	248 MB	PEFT LoRA adapter
`gguf/lima_dpo_adapter_v2_f16.gguf`	101 MB	GGUF LoRA for llama.cpp `--lora`
`tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`	32 MB	tokenizer (unchanged from base)

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

SatorTenet

Model Tree

Base

google/gemma-4-E2B-it

Adapter

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer