mafia-gemma-4-12B-it

Model Overview

This repository contains the merged Transformers model for mafia-gemma-4-12B-it. The model is intended to be used as the player policy inside a Mafia game agent. In our production game, it is paired with the HOLY GRAIL agent architecture and a separate Time-to-Talk moderator.

The model was trained to improve:

• legal JSON/action formatting;
• role-conditioned choices for Mafia, Detective, Doctor, and Villager;
• public discussion, accusation, defense, claim, and vote behavior;
• belief, claim, suspicion, and deception tracking;
• night actions such as Mafia kills, Doctor protects, and Detective checks;
• compatibility with moderated multi-day Mafia games using classic win conditions.

Training Data

Fine-tuning used the unified Alfaxad/mafia-dataset, a canonical event-log corpus built for social-deduction agents. The dataset combines converted and normalized examples from:

• Mini-Mafia style action primitives and role-conditioned decisions;
• LLMafia / Time-to-Talk style communication and timing data;
• Bayesian Social Deduction / GRAIL belief and role-count examples;
• Werewolf / Wolf-Enhance style debate traces after conversion to the Mafia schema;
• WOLF-inspired deception, suspicion, claim, and voting labels where available;
• our seven-player harness logs with 2 Mafia, 1 Detective, 1 Doctor, and 3 Villagers.

The schema separates public transcript, private role information, legal action sets, hidden state, votes, night actions, claims, and outcome labels. Hidden information is intentionally represented in the training rows only where the acting role is allowed to see it.

Fine-Tuning Recipe

• Base model: unsloth/gemma-4-12b-it
• Method: LoRA SFT with Unsloth and TRL
• LoRA: rank 32, alpha 64
• Context length: 4096 tokens
• Training sample: 60k train rows
• Validation/test sample: 2k validation rows, 2k test rows
• Optimizer steps: 1000
• Final eval loss: 0.57703
• Final train loss: 0.04144
• Deployment formats: merged Transformers weights and Q8_0 GGUF

How to Run

Install recent Transformers support for Gemma 4:

bash
Copy code
pip install -U "transformers>=5.11.0" accelerate torch sentencepiece protobuf

Run a text-only Mafia action prompt:

python
Copy code
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "Alfaxad/mafia-gemma-4-12B-it"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": (
            "You are a Mafia game agent. Use only legal public information, "
            "respect your private role, and return compact JSON."
        ),
    },
    {
        "role": "user",
        "content": (
            "Role: Detective. Alive players: Ada, Blake, Casey, Devon, Emery. "
            "Night 2 action: choose one player to investigate."
        ),
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    return_tensors="pt",
    return_dict=True,
    add_generation_prompt=True,
)
device = next(model.parameters()).device
inputs = {key: value.to(device) for key, value in inputs.items()}

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=True,
        temperature=1.0,
        top_p=0.95,
        top_k=64,
    )

print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

For the lightweight local runtime, use the GGUF repository: Alfaxad/mafia-gemma-4-12B-it-gguf.

Full-Game Evaluation

The table below combines the merged BF16 and GGUF Q8_0 results as one model family, mafia-gemma-4-12B-it. Results come from 24 full seven-player games, with every player using HOLY GRAIL agent architecture and the non-player moderator fixed to base Gemma 4 12B BF16 using a Time-to-Talk scheduler plus generator. The benchmark included 20 pairwise local-vs-frontier games and 4 mixed all-star games. Total API/player/moderator errors: 0.

Overall Slot Scoreboard

Pairwise Local-vs-Frontier Results

mafia-gemma-4-12B-it combines BF16 and Q8_0 rows. Each opponent has four local-side trials: two with Mafia Gemma controlling Mafia and two with Mafia Gemma controlling Good.

Role Diagnostics

Runtime Call Metrics

Latency is runtime and provider dependent, so use this as an operational reference rather than a pure capability ranking.

Intended Use

This model is intended for game agents, research harnesses, and AI-native social-deduction experiences. It works best when paired with:

• an authoritative game engine that enforces legal actions;
• strict public/private view isolation;
• role-specific prompts;
• an external memory/ledger layer such as HOLY GRAIL;
• a moderator that controls turn timing and table flow.

Limitations

• The full-game benchmark is small and game-specific. It should not be treated as a broad general reasoning benchmark.
• The model is not a security boundary. Hidden-role secrecy must be enforced by the engine and prompt construction.
• The model is tuned for Mafia-style game behavior, including in-game deception. Do not use it for real-world deception, impersonation, or misinformation.
• Best behavior depends on structured prompts and legal-action validation.

License

This model is distributed under the Apache 2.0 license, following the base Gemma 4 license information provided by the upstream model card.