GGUF
A GGUF version (for llama.cpp, Ollama, LM Studio) is available thanks to mradermacher:
mradermacher/gladios-tiny.story-0.1B-GGUF
🧠 Overview
gladios-tiny.story-0.1B is a decoder-only Transformer (GPT-2-style architecture, ~124M / 0.1B
parameters) trained from scratch on the full TinyStories
dataset (~2.1 million children's stories, ~470M tokens).
The model uses the GPT-2 tokenizer (50,257-token vocabulary) and generates short, coherent
stories in English, in the style of the training data.
👉 Try it without installing anything: Gradio demo (ZeroGPU)
👉 Full training code: github.com/plimb-ai/gladios-tiny-story
🏗️ Architecture
Table | |
|---|
| Type | Decoder-only Transformer (GPT-2 architecture) |
| Parameters | ~124M (0.1B) |
| Layers | 12 |
| Attention heads | 12 |
| Embedding dim | 768 |
| Context length | 512 tokens |
| Tokenizer | GPT-2 (tiktoken / 50,257-token vocab) |
| Weights | safetensors |
🚀 Usage
from transformers import pipeline gen = pipeline("text-generation", model="plimb/gladios-tiny.story-0.1B")print(gen( "Once upon a time", max_new_tokens=200, do_sample=True, temperature=0.8, top_k=50,)[0]["generated_text"])
Or directly with AutoModelForCausalLM:
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model = AutoModelForCausalLM.from_pretrained("plimb/gladios-tiny.story-0.1B")tok = AutoTokenizer.from_pretrained("plimb/gladios-tiny.story-0.1B") ids = tok("Once upon a time", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=200, do_sample=True, temperature=0.8, top_k=50, pad_token_id=tok.eos_token_id)print(tok.decode(out[0], skip_special_tokens=True))
📚 Training data
- Dataset: roneneldan/TinyStories (full
train split)
- Tokenizer: GPT-2, with each story separated by the special
<|endoftext|> token
- Trained on the entire dataset (no subsampling)
⚠️ Limitations
- Only writes short children's stories, in English
- No general world knowledge, no instruction-following, no conversational ability
- May hallucinate or lose coherence on prompts far outside the TinyStories style
📄 License
MIT