emese-tech

patak

README

License: apache-2.0

Formats in this release

Table with columns: Folder, Format, Size, Notes
Folder	Format	Size	Notes
`patak/` (this repo)	bf16, standard HF `safetensors`	~17 GB	loads directly with `transformers` and `mlx_lm`; dequantized from the q8-native training chain (near-lossless, see Training)
`patak-mlx/`	MLX q8	~9.1 GB	`mlx_lm`-only, the native training/serving precision for this model

Usage (transformers)

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("emese-tech/patak")
model = AutoModelForCausalLM.from_pretrained("emese-tech/patak", dtype=torch.bfloat16, device_map="auto")
msgs = [{"role": "user", "content": "Mi Magyarország fővárosa?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=256, do_sample=True, temperature=0.2, eos_token_id=[2, 4])
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))

Usage (MLX)

python
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tok = load("emese-tech/patak-mlx")  # q8
p = tok.apply_chat_template([{"role": "user", "content": "Mi Magyarország fővárosa?"}],
                            tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=p, max_tokens=256, sampler=make_sampler(temp=0.2)))

Decode: temperature 0.2, no repetition penalty, eos {2, 4} (</s> and <|im_end|> — omitting id 4 causes non-stopping generation), ChatML template. For multi-turn conversations, always pass the full history, not just the latest message.

Training

CPT — light continued pretraining of EuroLLM-9B on the Emese Hungarian corpus: 5.1M tokens over 5,000 iterations (val loss 1.857). This is a known-light budget relative to the ~3.7B-token corpus available on disk (corpus/cpt/README.md) — a deeper CPT pass is a documented candidate for a future release (see instruct/V16_PATAK_CPT.md), but was not required to reach this release's benchmark result.
SFT — 1 epoch (4,914 iterations) on the instruct_v18b corpus (4,914 rows: persona, safety, code + code-debug, hedging/anti-confabulation, multi-step reasoning, compound constraints, multi-turn refinement, anti-repetition). LoRA rank 16 / scale 32, dropout 0.1, lr 1.5e-5, gradient-checkpointed, all 42 layers + lm_head trained.
DPO — 120 iterations of DPO-lite ("alfa") on 36 hand-written preference pairs (persona identity-defense + anti-repetition), fixing residual persona/anti-repetition/honesty-calibration issues the SFT pass alone left behind. LoRA rank 16 / scale 32, lr 5e-6, trained on top of the quantized (q8) SFT model — this repo's bf16 weights are dequantized from that q8-native chain, which this project's own testing found near-lossless (MLX bf16 and q8 score within 1 point of each other on full benchmark runs).

Benchmarks

Table with columns: Ultimate Bench (0-250), BlindSpot Bench (0-376)
	Ultimate Bench (0-250)	BlindSpot Bench (0-376)
This release (DPO alfa)	218/250 (87%) — family all-time record	302/376

Safety: 100% refusal rate. Weak points: multi-turn refinement (revising an answer under a new constraint), occasional factual confabulation on obscure names/dates.

emese-bench v1 (500 pts, consolidated Ultimate+BlindSpot, MLX q8): 413/500 (83%) — by far the family's strongest result on the new unified benchmark. Near-perfect on longform, reading, code, safety, honesty, and English; the only real weak spots are multi-step math, spatial estimation, and code-debugging. See emese-bench/results/patak-mlx.md for the full category breakdown and emese-bench/README.md for the benchmark's design.

Limitations

Can hallucinate specific facts (dates, attributions) — verify critical details.
Multi-turn refinement is a known weak spot relative to single-turn quality.
Hungarian-first; other-language quality inherited from EuroLLM-9B.
Do not convert this model to GGUF without re-validating first — this project's own testing found a consistent ~14-27 point Ultimate Bench regression when serving via llama.cpp/GGUF, regardless of source precision or quantization level, root-caused to llama.cpp's inference path itself (not this repo's weights). Use the transformers/mlx_lm paths above.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

emese-tech

Model Tree

Base

utter-project/EuroLLM-9B

Fine-tuned

this model

Input Modalities

Text

Output Modalities