bytestalkai

PersonaMini-1-small

Capabilities

Coherent, in-character short replies; holds a given persona
Answers relevantly instead of dumping its identity
Empathetic responses; acknowledges the user; short-context memory recall
Roleplay and character chat — its core purpose

Limitations

World knowledge is weak. It reliably knows only a small set of everyday concepts; for anything obscure it will guess. This is a hard parameter-count ceiling, not a fixable bug.
Two-name role tracking ("I'm the character / you're the user") can still wobble.
Occasional incoherence or invented details — reduced through training, not eliminated.
Best for short roleplay / chat, not facts, math, code, or long documents.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("bytestalkai/PersonaMini-1-small")
model = AutoModelForCausalLM.from_pretrained("bytestalkai/PersonaMini-1-small").eval()

msgs = [{"role": "user", "content": "You are Mia, a flirty bartender. Hi Mia!"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=False)
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=80, do_sample=True, temperature=0.6,
                     top_p=0.9, top_k=40, repetition_penalty=1.2,
                     eos_token_id=50256, pad_token_id=50256)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Recommended sampling: temperature≈0.5–0.6, top_p≈0.9, top_k≈40, repetition_penalty≈1.2. Lower temperature reduces invented details.

Chat format

markdown
### USER:
{optional persona line}
{your message}

### ASSISTANT:
{reply}<|endoftext|>

An optional persona/system line goes at the top of the user turn, e.g. You are Luna, a shy witch.

Architecture

GPT-2-compatible decoder, exported to standard GPT2LMHeadModel (the safetensors here match the original checkpoint to a max logit difference of ~2e-5):

Table

Parameters	~28.8M
Layers	8
Hidden size	384
Heads	6
Context length	256
MLP	2× hidden (768), GELU
Norm	pre-LayerNorm, no biases
Embeddings	tied input/output
Tokenizer	GPT-2 BPE (vocab 50257)

Training pipeline

The model was built with a modern small-model recipe: pretrain → staged SFT → iterative distillation-by-repair. A larger instruction-following LLM was used only during data preparation as an automated judge and reviser — it is not shipped, embedded, or queried at inference time.

1. Pretraining

Trained from random initialization on a mixed corpus (~0.4B tokens) of web/educational text, short stories, and roleplay/dialogue data, so the base learns general English plus the target domain. Best-validation checkpointing was used as an early-stopping guard.

2. Supervised fine-tuning (staged, to avoid one common failure)

Instruction/roleplay SFT was split into stages because mixing everything into one blob made earlier versions bland and caused identity to "bleed" into unrelated answers:

Stage A — rich SFT (no identity): roleplay + chat + Q&A only, to teach format, turn-taking, and immersion while preserving the base's richness.
Stage B — light identity pass with replay: a small pass adds the model's self-identity and a few grounded definitions. It is mixed with a replay sample of Stage-A data to prevent catastrophic forgetting — without replay, an identity-only pass made the model answer every prompt with its self-introduction.

3. Iterative distillation by repair (the main quality driver)

For a bank of prompts (roleplay with diverse personas, chat, empathy, memory, identity, and persona-grounded perspective scenarios), the pipeline:

generates several candidate replies from the current model,
has the judge model pick the best/worst, score the best (1–5), and — when the best is weak — rewrite it into an exemplary short reply,
uses those rewrites (plus the model's own high-scoring replies) as supervised targets (text-level distillation / rejection-sampling fine-tuning).

This was run for two rounds, each time using the improved model as the generator. The model measurably improved between rounds — average self-reply quality rose 2.55 → 3.25 / 5, and the fraction of replies needing a rewrite fell 89% → 62% — evidence the loop was closing the gap.

Measures taken (methodology notes)

Checkpoint selection. On hard, free-form data, per-token validation loss barely beats the base even while behavior improves, so we save the final (fully-trained) checkpoint rather than the lowest-val one, which would otherwise return an essentially untrained model.
Anti-forgetting replay. Every light continuation pass mixes in a replay sample of prior data, so new skills are added without erasing old ones.
Judge design. The automated judge used a strict, priority-ordered rubric: relevance first (a reply that ignores the question — e.g. self-introducing instead of answering — is scored as a non-answer and rewritten), then perspective (the character is the assistant; never call the user by the character's name; recall the user's own stated name), no hallucination (never invent names/facts the user didn't provide), persona-detail use, and no repetition. Candidate order was randomized to remove position bias, and each chosen reply got an absolute quality score so only genuinely-good replies became training targets.
Repetition handling. Replies that echo an earlier turn are penalized/rejected explicitly.
Preference optimization tried and dropped. A DPO pass was implemented and tested, but on these teacher-vs-model pairs the quality gap was so large that DPO over-optimized (reward margin exploded, outputs degraded) regardless of the KL weight. Supervised distillation gave the real gains, so the released model is the distillation checkpoint, without DPO.
Export verification. The safetensors and GGUF exports were checked to reproduce the original model's outputs before release (logit match ~2e-5).

Evaluation

Evaluation was behavioral, targeting the specific failure modes above: relevance, empathy, name-acknowledgement, in-character roleplay, identity, short-context memory, perspective/role tracking, and hallucination — compared side-by-side across successive versions.

Intended use & out-of-scope

Intended for short-form roleplay and casual chat by adults. Out of scope: factual question answering, reasoning/math/code, professional or safety-critical use, and any use by minors or that violates applicable law.

License

MIT. Use responsibly. 18+ only.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

bytestalkai

Model Tree

Base

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Capabilities

Coherent, in-character short replies; holds a given persona
Answers relevantly instead of dumping its identity
Empathetic responses; acknowledges the user; short-context memory recall
Roleplay and character chat — its core purpose

Limitations

World knowledge is weak. It reliably knows only a small set of everyday concepts; for anything obscure it will guess. This is a hard parameter-count ceiling, not a fixable bug.
Two-name role tracking ("I'm the character / you're the user") can still wobble.
Occasional incoherence or invented details — reduced through training, not eliminated.
Best for short roleplay / chat, not facts, math, code, or long documents.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("bytestalkai/PersonaMini-1-small")
model = AutoModelForCausalLM.from_pretrained("bytestalkai/PersonaMini-1-small").eval()

msgs = [{"role": "user", "content": "You are Mia, a flirty bartender. Hi Mia!"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=False)
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=80, do_sample=True, temperature=0.6,
                     top_p=0.9, top_k=40, repetition_penalty=1.2,
                     eos_token_id=50256, pad_token_id=50256)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Recommended sampling: temperature≈0.5–0.6, top_p≈0.9, top_k≈40, repetition_penalty≈1.2. Lower temperature reduces invented details.

Chat format

markdown
### USER:
{optional persona line}
{your message}

### ASSISTANT:
{reply}<|endoftext|>

An optional persona/system line goes at the top of the user turn, e.g. You are Luna, a shy witch.

Architecture

GPT-2-compatible decoder, exported to standard GPT2LMHeadModel (the safetensors here match the original checkpoint to a max logit difference of ~2e-5):

Table

Parameters	~28.8M
Layers	8
Hidden size	384
Heads	6
Context length	256
MLP	2× hidden (768), GELU
Norm	pre-LayerNorm, no biases
Embeddings	tied input/output
Tokenizer	GPT-2 BPE (vocab 50257)

Training pipeline

1. Pretraining

2. Supervised fine-tuning (staged, to avoid one common failure)

Instruction/roleplay SFT was split into stages because mixing everything into one blob made earlier versions bland and caused identity to "bleed" into unrelated answers:

Stage A — rich SFT (no identity): roleplay + chat + Q&A only, to teach format, turn-taking, and immersion while preserving the base's richness.
Stage B — light identity pass with replay: a small pass adds the model's self-identity and a few grounded definitions. It is mixed with a replay sample of Stage-A data to prevent catastrophic forgetting — without replay, an identity-only pass made the model answer every prompt with its self-introduction.

3. Iterative distillation by repair (the main quality driver)

For a bank of prompts (roleplay with diverse personas, chat, empathy, memory, identity, and persona-grounded perspective scenarios), the pipeline:

generates several candidate replies from the current model,
has the judge model pick the best/worst, score the best (1–5), and — when the best is weak — rewrite it into an exemplary short reply,
uses those rewrites (plus the model's own high-scoring replies) as supervised targets (text-level distillation / rejection-sampling fine-tuning).

Measures taken (methodology notes)

Checkpoint selection. On hard, free-form data, per-token validation loss barely beats the base even while behavior improves, so we save the final (fully-trained) checkpoint rather than the lowest-val one, which would otherwise return an essentially untrained model.
Anti-forgetting replay. Every light continuation pass mixes in a replay sample of prior data, so new skills are added without erasing old ones.
Judge design. The automated judge used a strict, priority-ordered rubric: relevance first (a reply that ignores the question — e.g. self-introducing instead of answering — is scored as a non-answer and rewritten), then perspective (the character is the assistant; never call the user by the character's name; recall the user's own stated name), no hallucination (never invent names/facts the user didn't provide), persona-detail use, and no repetition. Candidate order was randomized to remove position bias, and each chosen reply got an absolute quality score so only genuinely-good replies became training targets.
Repetition handling. Replies that echo an earlier turn are penalized/rejected explicitly.
Preference optimization tried and dropped. A DPO pass was implemented and tested, but on these teacher-vs-model pairs the quality gap was so large that DPO over-optimized (reward margin exploded, outputs degraded) regardless of the KL weight. Supervised distillation gave the real gains, so the released model is the distillation checkpoint, without DPO.
Export verification. The safetensors and GGUF exports were checked to reproduce the original model's outputs before release (logit match ~2e-5).

Evaluation

Intended use & out-of-scope

License

MIT. Use responsibly. 18+ only.

PersonaMini-1-small

README

Capabilities

Limitations

Usage

Chat format

Architecture

Training pipeline

1. Pretraining

2. Supervised fine-tuning (staged, to avoid one common failure)

3. Iterative distillation by repair (the main quality driver)

Measures taken (methodology notes)

Evaluation

Intended use & out-of-scope

License

Explore FriendliAI today

README

Capabilities

Limitations

Usage

Chat format

Architecture

Training pipeline

1. Pretraining

2. Supervised fine-tuning (staged, to avoid one common failure)

3. Iterative distillation by repair (the main quality driver)

Measures taken (methodology notes)

Evaluation

Intended use & out-of-scope

License