ChessSLM API & Inference Endpoint

Overview

Architecture: GPT-2
Parameters: ~30M
Training data: 100k chess games
Notation: SAN (Standard Algebraic Notation)
Task: Autoregressive chess move generation

ChessSLM demonstrates that specialized small language models can perform competitively in narrow domains such as chess.

Capabilities

ChessSLM can play chess by generating moves sequentially in SAN notation.
It has been evaluated in matches against several language models, including:

Claude [Won against it]
Gemini [Lost again it]
Qwen
GPT-2
GPT-Neo
Pythia
LLaMA
Mistral
other small chess-oriented models

The model achieves an averaging rating of around ~1054 Elo against other language models despite its small size.

Benchmark Results

Table with columns: Model, Elo Rating
Model	Elo Rating
FlameF0X/ChessSLM	1154
DedeProGames/mini-chennus	1114
EleutherAI/pythia-70m-deduped	1099
nlpguy/smolchess-v2	1092
DedeProGames/dialochess	1078
nlpguy/amdchess-v9	1073
mlabonne/grandpythia-200k-70m	1065
FlameF0X/ChessSLM-PM	1055
DedeProGames/Chesser-248K-Mini

Limitations

Like many language-model-based chess systems, ChessSLM has several limitations:

Illegal move hallucinations: The model may occasionally generate moves that violate chess rules.
No board-state verification: Moves are generated purely from learned patterns rather than a validated game state.
Limited strategic depth: While competitive at lower Elo levels, it cannot match dedicated chess engines.

These limitations are common for pure language-model chess agents that do not use external rule engines.

Future Improvements

Potential improvements include:

Adding move legality filtering
Integrating board-state validation
Training on larger datasets
Reinforcement learning through self-play

Summary

ChessSLM shows that very small language models can achieve meaningful chess performance when trained on domain-specific data.
It serves as a lightweight baseline for exploring LLM-based chess agents and specialized small language models (SLMs).

Overview

Architecture: GPT-2
Parameters: ~30M
Training data: 100k chess games
Notation: SAN (Standard Algebraic Notation)
Task: Autoregressive chess move generation

ChessSLM demonstrates that specialized small language models can perform competitively in narrow domains such as chess.

Capabilities

ChessSLM can play chess by generating moves sequentially in SAN notation.
It has been evaluated in matches against several language models, including:

Claude [Won against it]
Gemini [Lost again it]
Qwen
GPT-2
GPT-Neo
Pythia
LLaMA
Mistral
other small chess-oriented models

The model achieves an averaging rating of around ~1054 Elo against other language models despite its small size.

Benchmark Results

Table with columns: Model, Elo Rating
Model	Elo Rating
FlameF0X/ChessSLM	1154
DedeProGames/mini-chennus	1114
EleutherAI/pythia-70m-deduped	1099
nlpguy/smolchess-v2	1092
DedeProGames/dialochess	1078
nlpguy/amdchess-v9	1073
mlabonne/grandpythia-200k-70m	1065
FlameF0X/ChessSLM-PM	1055
DedeProGames/Chesser-248K-Mini

Limitations

Like many language-model-based chess systems, ChessSLM has several limitations:

Illegal move hallucinations: The model may occasionally generate moves that violate chess rules.
No board-state verification: Moves are generated purely from learned patterns rather than a validated game state.
Limited strategic depth: While competitive at lower Elo levels, it cannot match dedicated chess engines.

These limitations are common for pure language-model chess agents that do not use external rule engines.

Future Improvements

Potential improvements include:

Adding move legality filtering
Integrating board-state validation
Training on larger datasets
Reinforcement learning through self-play

ChessSLM

README

Overview

Capabilities

Benchmark Results

Limitations

Future Improvements

Summary

Explore FriendliAI today

README

Overview

Capabilities

Benchmark Results

Limitations

Future Improvements

Summary