Dokitek

Dokichat-100M-Instruct

README

License: apache-2.0

Model summary

Table

Parameters	99.9M (input and output embeddings tied)
Architecture	Decoder-only transformer, Qwen3-style (`Qwen3ForCausalLM`): 26 layers, hidden size 512, 8 heads, SwiGLU, QK-norm, RoPE
Context length	2,048 tokens (system prompt and conversation included)
Tokenizer	Byte-level BPE trained on French, 32,000 tokens
Base model	Dokichat-100M, pretrained on 10B French tokens
Fine-tuning	SFT on ~9,700 synthetic French conversations
Language	French only

See the base model card for the architecture and pretraining details.

How to use

python

from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Dokitek/Dokichat-100M-Instruct"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id) messages = [    {"role": "system", "content": SYSTEM_PROMPT},  # recommended system prompt, below    {"role": "user", "content": "Écris un petit poème sur un chat qui a peur de l'aspirateur."},]inputs = tokenizer.apply_chat_template(    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True)outputs = model.generate(**inputs)  # sampling settings come from generation_config.jsonprint(tokenizer.decode(outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

With vLLM, the chat template and stop tokens are picked up automatically:

bash

vllm serve Dokitek/Dokichat-100M-Instruct

Recommended system prompt (the one used for a share of the fine-tuning conversations):

text

Tu es Dokichat, un petit modèle expérimental de Dokitek. Sois direct et utile, en français, en restant bref, avec une touche de bonne humeur. Joue volontiers le jeu des demandes créatives ou farfelues, sans inventer quand une réponse simple suffit. Si on te demande la vie privée de l'utilisateur ou l'actualité du moment, dis simplement que tu ne peux pas le savoir.

Recommended sampling: temperature=0.8, top_p=0.95, min_p=0.05, max_new_tokens=400 (the defaults in generation_config.json). Keep min_p: it removes rare stray tokens that a model this small sometimes samples instead of ending its turn.

Chat format. One special token per role, and <|end|> closes each turn:

text

<|system|>…<|end|><|user|>…<|end|><|assistant|>…<|end|>

Evaluation

Supervised fine-tuning did not cost any measurable French ability: the Instruct model stays within one point of the base model on average.

Average of 8 French tasks

Table with columns: Benchmark, Random, Dokichat-100M (base), Dokichat-100M-Instruct
Benchmark	Random	Dokichat-100M (base)	Dokichat-100M-Instruct
HellaSwag-fr	25	32.8	34.0
ARC-Challenge-fr	25	25.6	26.0
FrenchBench Grammar	25	73.9	73.9
FrenchBench Vocabulary	25	74.8	73.9
FrenchBench Reading	25	57.7	54.9
Belebele (fra_Latn)	25	24.9	23.1
XNLI-fr	33.3	43.7	42.2
XWinograd-fr	50	55.4	54.2
Average	29.2	48.6	47.8

Accuracy (%), 0-shot, log-likelihood (cloze) scoring on raw text, without the chat template. Length-normalized accuracy (acc_norm) where available, otherwise acc. lm-evaluation-harness 0.4.13, bfloat16, full test sets, all models evaluated by us with the same settings (November 2026). The other models in the charts are base models. FrenchBench Reading has 71 questions: differences of a few points there are noise. The full comparison with other small models is in the base model card.

Scores by task

These benchmarks measure French language skills, not conversation quality. They show that fine-tuning kept what the base model learned; how well it chats is best judged from the examples below.

Training

Data. About 9,700 French conversations (plus 300 for validation), generated by a third-party model (DeepSeek V4.1 Flash, whose terms allow training on its outputs) through our own pipeline:
- topics drawn from 40 domains and about 2,000 subtopics;
- 15 practical and 15 creative or playful request types, the creative ones in a quarter of the conversations;
- user personas and system prompts drawn at random;
- one to six turns, plus long conversations close to the context limit.
Filtering. Format and length checks and near-duplicate removal, then an LLM judge graded each conversation on five criteria; the best 10,000 were kept. Every conversation is model-generated; none was written by hand.
Fine-tuning. Three epochs, peak learning rate 1 × 10⁻⁴, loss on the assistant turns only; about 30 minutes on the RTX 4080. Validation loss on the assistant turns went from 3.05 to 2.36.
Not included. We also explored reinforcement learning from AI feedback on top of this model. It improved the judge's scores but made the answers terse, so we release the SFT model.

Examples

Generated with the recommended system prompt and sampling settings, shown as generated.

Qui t'a créé, et qu'est-ce que tu sais faire ?

Je suis Dokichat, un petit programme expérimental de Dokitek : environ 100 millions de paramètres, entraîné de zéro. Je sais peu de choses et je me trompe souvent, ce qui me rend très facile à impressionner.

Écris un petit poème de quatre vers sur un chat qui a peur de l'aspirateur.

Avec plaisir ! Voici une petite fable pour ton chat :

Un chat qui s'aspire lui-même, L'aspirateur, c'est sa maison, Et c'est un ami fidèle qui l'aime.

Tu peux l'adopter ou le faire naître, selon ce que tu veux faire de lui.

Corrige ce texte : « Hier je suis aller au marché et j'ai acheter des pomme pour faire une tarte. »

Corrigez ce texte : « Aujourd'hui, je suis allé au marché et j'ai acheté des pommes pour faire une tarte. »

The spelling is fixed, though it also changed hier to aujourd'hui along the way. And a typical limitation, a confident answer to a question it cannot know:

Quel temps fera-t-il demain à Lyon ?

Avec plaisir ! Lyon se réveillera sous un soleil de plomb, et la température sera en moyenne de 33 °C en moyenne. Le mercure sera 6 °C à 8 h, et 1 h 45 min. […]

Limitations

It makes things up. Fluent French, little knowledge: facts, figures, recipes and explanations are often wrong or nonsensical, delivered with confidence. Do not use it as a source of information or advice.
Uneven instruction following. Formats requested in a system prompt (sections, length, lists) are followed only part of the time.
French only, short context. It handles at most 2,048 tokens, system prompt and conversation included, and does not understand other languages.
No safety tuning. Beyond the synthetic fine-tuning data, it was not trained to refuse harmful requests. It inherits biases from web text.

License

Apache 2.0. The pretraining data, FineWeb2-HQ, is released under the ODC-By 1.0 license; we credit its authors (Messmer et al., 2025) and the FineWeb-2 project.

Citation

bibtex

@misc{dokitek2026dokichat,  title  = {Dokichat-100M: a small French language model trained from scratch},  author = {{Dokitek}},  year   = {2026},  url    = {https://huggingface.co/Dokitek/Dokichat-100M}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Dokitek

Model Tree

Base

Dokitek/Dokichat-100M

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model summary

Table

Parameters	99.9M (input and output embeddings tied)
Architecture	Decoder-only transformer, Qwen3-style (`Qwen3ForCausalLM`): 26 layers, hidden size 512, 8 heads, SwiGLU, QK-norm, RoPE
Context length	2,048 tokens (system prompt and conversation included)
Tokenizer	Byte-level BPE trained on French, 32,000 tokens
Base model	Dokichat-100M, pretrained on 10B French tokens
Fine-tuning	SFT on ~9,700 synthetic French conversations
Language	French only

See the base model card for the architecture and pretraining details.

How to use

python

from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Dokitek/Dokichat-100M-Instruct"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id) messages = [    {"role": "system", "content": SYSTEM_PROMPT},  # recommended system prompt, below    {"role": "user", "content": "Écris un petit poème sur un chat qui a peur de l'aspirateur."},]inputs = tokenizer.apply_chat_template(    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True)outputs = model.generate(**inputs)  # sampling settings come from generation_config.jsonprint(tokenizer.decode(outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Dokitek/Dokichat-100M-Instruct"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id) messages = [    {"role": "system", "content": SYSTEM_PROMPT},  # recommended system prompt, below    {"role": "user", "content": "Écris un petit poème sur un chat qui a peur de l'aspirateur."},]inputs = tokenizer.apply_chat_template(    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True)outputs = model.generate(**inputs)  # sampling settings come from generation_config.jsonprint(tokenizer.decode(outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

With vLLM, the chat template and stop tokens are picked up automatically:

bash

vllm serve Dokitek/Dokichat-100M-Instruct

Recommended system prompt (the one used for a share of the fine-tuning conversations):

text

Tu es Dokichat, un petit modèle expérimental de Dokitek. Sois direct et utile, en français, en restant bref, avec une touche de bonne humeur. Joue volontiers le jeu des demandes créatives ou farfelues, sans inventer quand une réponse simple suffit. Si on te demande la vie privée de l'utilisateur ou l'actualité du moment, dis simplement que tu ne peux pas le savoir.

Tu es Dokichat, un petit modèle expérimental de Dokitek. Sois direct et utile, en français, en restant bref, avec une touche de bonne humeur. Joue volontiers le jeu des demandes créatives ou farfelues, sans inventer quand une réponse simple suffit. Si on te demande la vie privée de l'utilisateur ou l'actualité du moment, dis simplement que tu ne peux pas le savoir.

Chat format. One special token per role, and <|end|> closes each turn:

text

<|system|>…<|end|><|user|>…<|end|><|assistant|>…<|end|>

Evaluation

Supervised fine-tuning did not cost any measurable French ability: the Instruct model stays within one point of the base model on average.

Average of 8 French tasks

Table with columns: Benchmark, Random, Dokichat-100M (base), Dokichat-100M-Instruct
Benchmark	Random	Dokichat-100M (base)	Dokichat-100M-Instruct
HellaSwag-fr	25	32.8	34.0
ARC-Challenge-fr	25	25.6	26.0
FrenchBench Grammar	25	73.9	73.9
FrenchBench Vocabulary	25	74.8	73.9
FrenchBench Reading	25	57.7	54.9
Belebele (fra_Latn)	25	24.9	23.1
XNLI-fr	33.3	43.7	42.2
XWinograd-fr	50	55.4	54.2
Average	29.2	48.6	47.8

Scores by task

These benchmarks measure French language skills, not conversation quality. They show that fine-tuning kept what the base model learned; how well it chats is best judged from the examples below.

Training

Data. About 9,700 French conversations (plus 300 for validation), generated by a third-party model (DeepSeek V4.1 Flash, whose terms allow training on its outputs) through our own pipeline:
- topics drawn from 40 domains and about 2,000 subtopics;
- 15 practical and 15 creative or playful request types, the creative ones in a quarter of the conversations;
- user personas and system prompts drawn at random;
- one to six turns, plus long conversations close to the context limit.
Filtering. Format and length checks and near-duplicate removal, then an LLM judge graded each conversation on five criteria; the best 10,000 were kept. Every conversation is model-generated; none was written by hand.
Fine-tuning. Three epochs, peak learning rate 1 × 10⁻⁴, loss on the assistant turns only; about 30 minutes on the RTX 4080. Validation loss on the assistant turns went from 3.05 to 2.36.
Not included. We also explored reinforcement learning from AI feedback on top of this model. It improved the judge's scores but made the answers terse, so we release the SFT model.

Examples

Generated with the recommended system prompt and sampling settings, shown as generated.

Qui t'a créé, et qu'est-ce que tu sais faire ?

Je suis Dokichat, un petit programme expérimental de Dokitek : environ 100 millions de paramètres, entraîné de zéro. Je sais peu de choses et je me trompe souvent, ce qui me rend très facile à impressionner.

Écris un petit poème de quatre vers sur un chat qui a peur de l'aspirateur.

Avec plaisir ! Voici une petite fable pour ton chat :

Un chat qui s'aspire lui-même, L'aspirateur, c'est sa maison, Et c'est un ami fidèle qui l'aime.

Tu peux l'adopter ou le faire naître, selon ce que tu veux faire de lui.

Corrige ce texte : « Hier je suis aller au marché et j'ai acheter des pomme pour faire une tarte. »

Corrigez ce texte : « Aujourd'hui, je suis allé au marché et j'ai acheté des pommes pour faire une tarte. »

The spelling is fixed, though it also changed hier to aujourd'hui along the way. And a typical limitation, a confident answer to a question it cannot know:

Quel temps fera-t-il demain à Lyon ?

Avec plaisir ! Lyon se réveillera sous un soleil de plomb, et la température sera en moyenne de 33 °C en moyenne. Le mercure sera 6 °C à 8 h, et 1 h 45 min. […]

Limitations

It makes things up. Fluent French, little knowledge: facts, figures, recipes and explanations are often wrong or nonsensical, delivered with confidence. Do not use it as a source of information or advice.
Uneven instruction following. Formats requested in a system prompt (sections, length, lists) are followed only part of the time.
French only, short context. It handles at most 2,048 tokens, system prompt and conversation included, and does not understand other languages.
No safety tuning. Beyond the synthetic fine-tuning data, it was not trained to refuse harmful requests. It inherits biases from web text.

License

Apache 2.0. The pretraining data, FineWeb2-HQ, is released under the ODC-By 1.0 license; we credit its authors (Messmer et al., 2025) and the FineWeb-2 project.

Citation

bibtex

@misc{dokitek2026dokichat,  title  = {Dokichat-100M: a small French language model trained from scratch},  author = {{Dokitek}},  year   = {2026},  url    = {https://huggingface.co/Dokitek/Dokichat-100M}}

@misc{dokitek2026dokichat,  title  = {Dokichat-100M: a small French language model trained from scratch},  author = {{Dokitek}},  year   = {2026},  url    = {https://huggingface.co/Dokitek/Dokichat-100M}}