GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
GitMarco27
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
GitMarco27
Model Tree
mii-llm/zagreus-0.4B-ita
Input Modalities
Output Modalities
Supported Functionality
| Evaluation | Setting | Questions | Accuracy |
|---|---|---|---|
ITALIC official run_eval.py | zero-shot, fast=true, temperature 0 | 10,000 | 0.3381 |
| Public domain development set | zero-shot, greedy | 480 | 0.5813 |
| KL held-out set | first-token letter accuracy | 924 records after filtering | 0.6190 |
The official evaluator is pinned to commit 92df420ff686babeea54e217b9f90f8471374916. Zero-shot evaluation is
configured by setting few_shot_file: null in the official ITALIC configuration. This differs from the
repository's five-shot default and was accepted for this challenge submission.
For each answer option, the base model produces a domain-conditional cloze-PMI score:
text
s_i = log P(option_i | question) - log P(option_i | domain prompt)q = softmax(s / tau)
Two balanced option permutations are created for each question. At the first generation position, the model is trained with the following loss:
text
L = KL(q || softmax(z_ABCD))+ lambda * CE(z_ABCD, gold)+ mu * (-log P(first token is A, B, C, or D))
The final hyperparameters are tau=1, lambda=1, mu=0.1, learning rate 1e-4, batch size 16, five epochs,
weight decay 0.01, and seed 42. The teacher and student both start from the mandatory base model. No external
teacher checkpoint or generation API is used.
The 21,847-question source pool contains:
efederici/pinocchio;mik3ml/italian-dictionary;fraug-library/synonyms_dictionnaries.The KL build selects 5,000 training questions and 500 question-disjoint held-out questions, then creates two permutations per question. The 256-token prompt filter leaves 9,294 training records and 924 held-out records.
The source pool is deduplicated and checked against all ITALIC questions with exact normalized matching, MinHash LSH, and RapidFuzz confirmation. The final audit found no question overlap. ITALIC labels are not used for training.
The dataset repository records the original terms for each source. The Hugging Face card for the synonym mirror does not declare a license, so users should review its upstream terms before redistributing derived data.
This checkpoint is specialized for four-option multiple-choice questions. The following example uses the same prompt style as the official fast evaluation:
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "GitMarco27/zagreus-0.4b-italic-kl"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")messages = [{"role": "system", "content": "Sei un assistente utile."},{"role": "user","content": ("Rispondi alla seguente domanda a scelta multipla. ""Scrivi solo la lettera corretta.\n\n""Qual è la capitale d'Italia?\n\n""A) Milano\nB) Roma\nC) Torino\nD) Napoli\n\nRisposta:"),},]inputs = tokenizer.apply_chat_template(messages,add_generation_prompt=True,return_tensors="pt",return_dict=True,).to(model.device)output = model.generate(**inputs, do_sample=False, max_new_tokens=1)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
The final run used one NVIDIA A100 80GB PCIe GPU. The allocation lasted 0.7531 hours, including setup, target generation, training, evaluation, transfer, and verification. Based on the recorded provider price ratio, this equals 0.3622 H100-equivalent hours.
The immutable base revision is f463a5beabf054809ee3cc702c711215a5fde690. The model weights have SHA-256
5e0284467f727f22000b897a3c40f3e5c24b519de463f71a4212149fe1913c6e.
This is a benchmark-specific model, not a general instruction model. Training directly supervises the first
generated token, so later tokens may repeat. For normal multiple-choice inference, use greedy decoding with
max_new_tokens=1.
The model has about 438 million parameters. Its training data covers lexical, cultural, and civic questions better than syntax, orthography, and morphology.
Development of this project relied on AI assistants: Claude Fable 5, GPT-5.6-sol, Claude Opus, and Qwen3.6 27B. They supported code writing, experiment analysis, and documentation. The assistants were not used to generate training data: all KL targets were produced with the mandatory base model itself, as described above, and no external API was used for synthetic generation.