GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Kysaddy
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Kysaddy
Model Tree
Qwen/Qwen2.5-Coder-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Property | Value |
|---|---|
| Model name | Koda Standalone |
| Base checkpoint | Qwen/Qwen2.5-Coder-1.5B-Instruct |
| Model type | Causal Transformer language model |
| Base parameter count | Approximately 1.54B |
| Stored model | One merged model.safetensors checkpoint |
| Quantization | 4-bit NF4 merged checkpoint |
| Adapter required at inference | No |
| PEFT required at inference | No |
| Tokenizer included | Yes |
| Chat template included | Yes |
| Context length | 32,768 tokens for the base architecture |
| Published training corpus | 120 curated coding and chat examples |
| Published training run | 120 steps, learning rate 3e-5, maximum sequence length 64 |
| License | Apache-2.0 terms for the project artifact; review upstream Qwen terms |
The output checkpoint is approximately 1.1 GB in this 4-bit format. It contains the merged model weights, tokenizer, generation configuration, chat template, and model configuration. It intentionally does not contain adapter_model.safetensors or adapter_config.json.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerMODEL_ID = "Kysaddy/koda-qwen15"tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)model = AutoModelForCausalLM.from_pretrained(MODEL_ID,device_map="auto",low_cpu_mem_usage=True,)model.eval()messages = [{"role": "system","content": "You are Koda, a direct, accurate, and practical coding assistant.",},{"role": "user","content": "Write a Python function for binary search.",},]prompt = tokenizer.apply_chat_template(messages,tokenize=False,add_generation_prompt=True,)inputs = tokenizer(prompt, return_tensors="pt").to(model.device)with torch.no_grad():outputs = model.generate(**inputs,max_new_tokens=256,do_sample=False,pad_token_id=tokenizer.eos_token_id,)new_tokens = outputs[0, inputs["input_ids"].shape[1]:]print(tokenizer.decode(new_tokens, skip_special_tokens=True).strip())
For a CPU-only machine, use device_map="cpu". The 4-bit checkpoint still requires sufficient RAM and CPU generation may be slow.
python
from huggingface_hub import snapshot_downloadlocal_dir = snapshot_download(repo_id="Kysaddy/koda-qwen15",repo_type="model",local_dir="./models/koda-qwen15",)print(local_dir)
Then replace MODEL_ID with ./models/koda-qwen15.
The standalone checkpoint can be loaded by any Transformers-compatible server because it is a normal model directory and has no PEFT dependency. A local FastAPI service can load it directly with AutoModelForCausalLM.from_pretrained.
A generic requests client sends JSON to an OpenAI-compatible server as follows:
python
import requestsresponse = requests.post("http://127.0.0.1:8000/v1/chat/completions",json={"messages": [{"role": "user", "content": "Explain SQL injection and prevention."}],"max_new_tokens": 128,"do_sample": False,},timeout=180,)response.raise_for_status()print(response.json())
Koda is intended for coding assistance and technical chat, including Python, JavaScript, TypeScript, SQL, Git, Docker, APIs, debugging, testing, systems concepts, security explanations, uncertainty handling, and general software-engineering questions.
This is a compact model with a narrow fine-tuning corpus. Free-form responses may contain incomplete code, incorrect assumptions, stale APIs, or factual errors. Review, test, lint, and security-check generated output before using it. Do not expose secrets in prompts or execute untrusted generated code on production systems.