GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
temper-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
temper-ai
Model Tree
Qwen/Qwen2.5-Coder-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
HumanEval-Rust (multiple-rs from MultiPL-E, 156 tasks), %. pass@1 is the share of tasks solved on the first try, pass@10 the share solved in at least one of 10 tries.
| Model | Size | pass@1 | pass@10 |
|---|---|---|---|
| Qwen2.5-Coder-0.5B | 0.5B | 16.0 | 27.9 |
| StarCoder2-3B | 3B | 25.1 | 37.8 |
| Temper-1-0.5B | 0.5B | 25.2 | 38.0 |
| Qwen2.5-Coder-1.5B | 1.5B | 39.3 | 56.6 |
| Qwen2.5-Coder-3B | 3B | 44.7 | 67.4 |
| Temper-1-1.5B | 1.5B | 50.5 | 67.0 |
| Qwen3.5-9B | 9B | 56.7 | 73.0 |
| Qwen2.5-Coder-7B | 7B | 57.3 | 74.3 |
Every model ran through bigcode-evaluation-harness at commit 8fc5bae with the BigCode Models Leaderboard settings. The model completes code without a chat template, with temperature 0.2, top_p 0.95, 50 samples per task and 512 tokens including the prompt. Qwen2.5-Coder and StarCoder2 are the base versions. Qwen3.5-9B is the post-trained version. StarCoder2-3B scores 25.1 here and 24.5 on the leaderboard.
The Temper-1-0.5B card reports HumanEval-RS with the task sent as a chat message, so its numbers there differ from this table.
python
from transformers import AutoModelForCausalLM, AutoTokenizer name = "temper-ai/Temper-1-1.5B"tokenizer = AutoTokenizer.from_pretrained(name)model = AutoModelForCausalLM.from_pretrained(name, dtype="auto", device_map="auto") messages = [{"role": "user", "content": "Write a Rust function that returns the n-th Fibonacci number."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=512)print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))The default generation config samples with temperature 0.7, top_p 0.8, top_k 20 and repetition penalty 1.1.
A GGUF version for llama.cpp, LM Studio and Ollama is in temper-ai/Temper-1-1.5B-GGUF.
With ten tries Temper-1-1.5B solves 67.0% of tasks, about the same as Qwen2.5-Coder-3B with 67.4%. The model was trained only on Rust tasks with requests in English. Requests in other languages and questions unrelated to Rust may get made-up answers or refusals. It was not trained for fill-in-the-middle, so it is not a drop-in model for inline completion in an editor. The training data has few tasks on async code and web frameworks.
Apache 2.0, same as Qwen2.5-Coder-1.5B-Instruct.
from transformers import AutoModelForCausalLM, AutoTokenizer name = "temper-ai/Temper-1-1.5B"tokenizer = AutoTokenizer.from_pretrained(name)model = AutoModelForCausalLM.from_pretrained(name, dtype="auto", device_map="auto") messages = [{"role": "user", "content": "Write a Rust function that returns the n-th Fibonacci number."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=512)print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))