GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
beardymcgee
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
beardymcgee
Model Tree
Input Modalities
Output Modalities
Supported Functionality
This model is intended for:
It is not intended for factual question answering, safety-critical use, professional advice, or production assistant use.
The model was trained on a local GPT-2-tokenised FineWeb-Edu dataset.
FineWeb-Edu is an educational-text subset filtered from FineWeb. The FineWeb-Edu dataset card describes it as containing 1.3T tokens, with an additional 5.4T-token score-2 version, and lists its licence as Open Data Commons Attribution License ODC-By v1.0, subject to Common Crawl terms.
This model was trained in two main phases:
The final released checkpoint is the best validation checkpoint from the continuation run.
Previous 2B checkpoint validation loss:
text
3.3519Final 5B checkpoint validation loss:
text
3.2245991230010986Approximate validation perplexity:
text
exp(3.2246) ≈ 25.1python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM model_id = "beardymcgee/cascamini-gpt2-210m-edu-5b" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, dtype=torch.float16,).to("cuda") prompt = "The main purpose of an operating system is"inputs = tokenizer(prompt, return_tensors="pt").to("cuda") with torch.no_grad(): output = model.generate( **inputs, max_new_tokens=160, do_sample=True, temperature=0.55, top_k=50, top_p=0.9, repetition_penalty=1.1, no_repeat_ngram_size=4, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id, ) print(tokenizer.decode(output[0], skip_special_tokens=True))This model is small and was trained as a learning project. It is not instruction tuned.
Known issues include:
CascaMini-GPT2-210M-Edu-5B is the first packaged model in the CascaMini series. The next planned model is a Llama-style under-400M parameter model using a more modern architecture.
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM model_id = "beardymcgee/cascamini-gpt2-210m-edu-5b" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, dtype=torch.float16,).to("cuda") prompt = "The main purpose of an operating system is"inputs = tokenizer(prompt, return_tensors="pt").to("cuda") with torch.no_grad(): output = model.generate( **inputs, max_new_tokens=160, do_sample=True, temperature=0.55, top_k=50, top_p=0.9, repetition_penalty=1.1, no_repeat_ngram_size=4, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id, ) print(tokenizer.decode(output[0], skip_special_tokens=True))