GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
harpertoken
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
harpertoken
Model Tree
Input Modalities
Output Modalities
Supported Functionality
python
from transformers import pipeline generator = pipeline("text-generation", model="harpertoken/word")print(generator("The quick brown fox", max_new_tokens=60)[0]["generated_text"])WikiText-2 is about two million tokens drawn from Wikipedia verification articles. Training on it for a few epochs produces a model that reproduces that register, namely encyclopaedic and declarative English, and little else. The previous card reported three epochs at batch size one completed in roughly ten minutes on CPU or MPS; for a 124M-parameter model over that corpus at that batch size the arithmetic does not work, so I have left the training time unstated rather than repeat it.
There are no evaluation figures. Perplexity on WikiText-2 would be the obvious measurement and has not been recorded here. Treat this as an architectural and training demonstration: it is a working GPT-2 implementation at small scale, not a language model to build on. Concretely, generation collapses to end-of-text: the end token is the top prediction on ordinary prompts, with measured probability 0.31 to 0.93, so greedy decoding returns the prompt unchanged and sampling usually stops within a token or two. The usage example above will print the prompt back. It has no instruction tuning, no dialogue behaviour, and no pretraining beyond a narrow slice of one corpus.
The architecture follows Radford et al., Improving Language Understanding by Generative Pre-Training (2018). WikiText-2 is described in Merity et al., Pointer Sentinel Mixture Models (2016).