GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ahmedsamirtarjama
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ahmedsamirtarjama
Model Tree
Qwen/Qwen2.5-Coder-0.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
NL2SH-ALFA test set (300 tasks), scored with the
InterCode-ALFA harness: the command runs in Docker and an LLM judge (qwen3.8 via Ollama) compares it with the
reference. Same harness and judge for every row.
| Model | Accuracy | Easy | Medium | Hard |
|---|---|---|---|---|
| This model + sandbox vote + fallback (shellsafe CLI) | 53.7% | 0.70 | 0.51 | 0.40 |
| This model, greedy | 50.0% | |||
| Qwen3-Coder-30B-A3B-Instruct (teacher, for reference) | 57.7% | |||
| Qwen2.5-Coder-0.5B-Instruct, 8 examples in the prompt | 28.0% | 0.47 | 0.22 | 0.15 |
| Qwen2.5-Coder-0.5B-Instruct, zero-shot | 25.3% | |||
| SmolLCoder-360M-Instruct | 10.3% | 0.14 | 0.11 | 0.06 |
| functiongemma-270m-it | 0–0.7% |
"Sandbox vote" samples 1 greedy and 5 answers at temperature 0.7, runs them in a read-only sandbox and picks the answer whose output most others agree with. "Fallback" replaces a winner that fails with a bad flag, a missing tool or a syntax error. Scores were measured with the Q8_0 GGUF in llama.cpp; Q4_K_M was not benchmarked.
| File | Use |
|---|---|
model.safetensors + tokenizer/config | transformers, bf16 |
qwen05-v45-Q8_0.gguf | llama.cpp / llama-cpp-python, the evaluated build |
qwen05-v45-Q4_K_M.gguf | smaller and faster, not benchmarked |
The model was trained on exactly this format; other system prompts work less well.
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "ahmedsamirtarjama/shellsafe-qwen2.5-coder-0.5b"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo)messages = [ {"role": "system", "content": "You are a bash command generator. Given a natural language request, " "output only the bash command that accomplishes it. No explanation, no markdown fences."}, {"role": "user", "content": "Request: find all .log files bigger than 10MB under /var"},]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True)out = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))With llama.cpp: llama-cli -m qwen05-v45-Q8_0.gguf --jinja -sys "<system prompt above>" -p "Request: ...".
v4 (72k rows), v5 on config v5 (78k rows); the release is the average of their weights.from transformers import AutoModelForCausalLM, AutoTokenizer repo = "ahmedsamirtarjama/shellsafe-qwen2.5-coder-0.5b"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo)messages = [ {"role": "system", "content": "You are a bash command generator. Given a natural language request, " "output only the bash command that accomplishes it. No explanation, no markdown fences."}, {"role": "user", "content": "Request: find all .log files bigger than 10MB under /var"},]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True)out = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))