Overview
- Architecture: GPT-2
- Parameters: ~30M
- Training data: 441k chess games rated over 2000 elo
- Notation: SAN (Standard Algebraic Notation)
- Task: Autoregressive chess move generation
ChessSLM demonstrates that specialized small language models can perform competitively in narrow domains such as chess.
Capabilities
ChessSLM can play chess by generating moves sequentially in SAN notation.
It has been evaluated in matches against several language models, including:
- Claude [Won against it]
- Gemini [Lost again it]
- Qwen
- GPT-2
- GPT-Neo
- Pythia
- LLaMA
- Mistral
- other small chess-oriented models
The model achieves an averaging rating of around {tba} Elo against other language models despite its small size.
Table with columns: Model, Elo Rating| Model | Elo Rating |
|---|
| FlameF0X/ChessSLM | 1154 |
| DedeProGames/mini-chennus | 1114 |
| EleutherAI/pythia-70m-deduped | 1099 |
| nlpguy/smolchess-v2 | 1092 |
| DedeProGames/dialochess | 1078 |
| nlpguy/amdchess-v9 | 1073 |
| mlabonne/grandpythia-200k-70m | 1065 |
| FlameF0X/ChessSLM-PM | 1055 |
| DedeProGames/Chesser-248K-Mini |
Limitations
Like many language-model-based chess systems, ChessSLM has several limitations:
- Illegal move hallucinations: The model may occasionally generate moves that violate chess rules.
- No board-state verification: Moves are generated purely from learned patterns rather than a validated game state.
- Limited strategic depth: While competitive at lower Elo levels, it cannot match dedicated chess engines.
These limitations are common for pure language-model chess agents that do not use external rule engines.
Summary
ChessSLM shows that very small language models can achieve meaningful chess performance when trained on domain-specific data.
It serves as a lightweight baseline for exploring LLM-based chess agents and specialized small language models (SLMs).