GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Henry665
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Henry665
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Item | Value |
|---|---|
| Parameters | ~501M |
| Architecture | 24-layer decoder, hidden 1280, GQA 4:1 (20Q / 5KV) |
| Context length | 4096 tokens |
| Vocab size | 65536 (BPE) |
| FFN | SwiGLU, d_ffn=3456 |
| Position encoding | RoPE (θ=5×10⁵) |
| Training stages | Pretrain → SFT → DAPO RL |
| Best RL method | DAPO (Direct Advantage Policy Optimization) |
Evaluated on HumanEval, HumanEval+, MBPP, MBPP+ with n=8, temperature=0.2, top_p=0.95, sandbox execution.
| Metric | Score |
|---|---|
| pass@1 | 37.6% |
| pass@4 | 43.6% |
| pass@8 | 46.6% |
| Dataset | pass@1 |
|---|---|
| HumanEval+ | 34.1% |
| HumanEval | 36.1% |
| MBPP | 42.1% |
| MBPP+ | 38.3% |
| Model | Macro pass@1 |
|---|---|
| Qwen2.5-0.5B-Instruct | 36.7% |
| Walkie-Code-0.5B (this) | 38.4% (+1.7 pp) |
| Stage | pass@1 | pass@8 |
|---|---|---|
| SFT | 33.7% | 41.4% |
| DAPO (this checkpoint) | 37.6% | 46.6% |
Pretraining (~17B tokens, code-heavy ~70%)
SFT
RL (DAPO)
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "Henry665/Walkie-Code-0.5B" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,) prompt = "user: Write a Python function to check if a number is prime.\nassistant:"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2, top_p=0.95)print(tokenizer.decode(outputs[0], skip_special_tokens=True))bash
vllm serve Henry665/Walkie-Code-0.5B --dtype auto --max-model-len 4096Training used a simple dialog template:
text
user: <instruction>assistant: <python code>For code benchmarks, plain user: / assistant: text prompts are recommended.
bibtex
@misc{walkie-code-0.5b, title={Walkie-Code-0.5B: A Modular 0.5B Python Code LLM}, author={LLM Walk-Through Team}, year={2026}, howpublished={\url{https://huggingface.co/Henry665/Walkie-Code-0.5B}}}Apache 2.0
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "Henry665/Walkie-Code-0.5B" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,) prompt = "user: Write a Python function to check if a number is prime.\nassistant:"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2, top_p=0.95)print(tokenizer.decode(outputs[0], skip_special_tokens=True))vllm serve Henry665/Walkie-Code-0.5B --dtype auto --max-model-len 4096@misc{walkie-code-0.5b, title={Walkie-Code-0.5B: A Modular 0.5B Python Code LLM}, author={LLM Walk-Through Team}, year={2026}, howpublished={\url{https://huggingface.co/Henry665/Walkie-Code-0.5B}}}