GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Siddh07ETH
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Siddh07ETH
Model Tree
Base
Qwen/Qwen2.5-Coder-0.5B-Instruct
Fine-tuned
this model
Input Modalities
Text
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
Container
Atlas-Coder-2-0.5B is the flagship model of the Pluto AI research project by Siddharth N.R. — the second generation of the Atlas-Coder series and the most focused coding model released under the Pluto AI brand to date.
Built on top of Qwen2.5-Coder-0.5B-Instruct, Atlas-Coder-2 is trained exclusively on 50K execution-verified OSS-Instruct samples — real open-source Python functions that have been independently verified to execute correctly. This single-source, high-purity data strategy maximizes alignment with HumanEval+ and MBPP+ benchmark formats while keeping the training signal clean and consistent.
Unlike V1 which trained from the base model and used a 4-source mixture, Atlas-Coder-2 starts from an instruction-tuned foundation and specializes it further on execution-verified code. The result is a sharper, more reliable code generator with a lower hallucination rate on self-contained Python tasks.
Research Goal: Demonstrate that a sub-500M parameter model, fine-tuned exclusively on execution-verified code in a single Kaggle session, can match or exceed the coding performance of officially released instruct variants and outperform models up to 3× its parameter count.
Competitor scores from official technical reports.
GGUF Quantizations
| Property | Atlas-Coder-0.5B (V1) | Atlas-Coder-2-0.5B (V2) |
|---|---|---|
| Base model | Qwen2.5-Coder-0.5B Base | Qwen2.5-Coder-0.5B Instruct |
| Data sources | 4 (Magicoder + OSS + CodeFeedback + TACO) | 1 (OSS-Instruct exec-verified only) |
| Training samples | ~80K (mixed quality) | 50K (100% exec-verified) |
| LoRA rank | r=64 | r=32 (faster, leaner) |
| Epochs | 3 | 1 (instruct base needs less) |
The key architectural insight of V2: starting from an instruct model means the model already knows how to follow instructions and stop generating. V2 doesn't need to re-learn conversation structure — it only needs to deepen its Python code generation capability. This allows a single clean epoch on a smaller, higher-quality dataset to outperform a longer multi-epoch run on a noisier mixture.
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-Coder-0.5B-Instruct |
| Parameters | ~494 Million |
| Method | QLoRA (4-bit NF4 + LoRA) |
| LoRA Rank | r=32, α=64 |
| LoRA Dropout | 0.05 |
| LoRA Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Trainable Parameters | 17,596,416 |
| Training Epochs | 1 |
| Dataset | Samples | Why This Dataset |
|---|---|---|
| bigcode/self-oss-instruct-sc2-exec-filter-50k | 50,154 (train) + 508 (eval) | 100% execution-verified. Generated from real open-source Python. Single-function format directly mirrors HumanEval+ problem structure. No hallucinated solutions — every completion has been independently run and confirmed correct. |
Why single-source? The V1 multi-dataset mixture introduced noise from TACO (competitive programming verbosity) and CodeFeedback (multi-turn debug style), both of which poorly align with HumanEval+ single-function completion format. V2 eliminates this noise entirely. The OSS-Instruct exec-filtered dataset is already the highest-ROI data source for HumanEval+ performance — using 50K samples of it exclusively produces a cleaner gradient signal than mixing 80K samples of heterogeneous quality.
1. Instruct base = faster convergence
Starting from Qwen2.5-Coder-0.5B-Instruct means the ChatML format, stop-token behavior, and instruction-following discipline are already in place. The model only needs to deepen code generation quality — not learn conversation structure from scratch. This makes 1 epoch sufficient where V1 needed 3.
2. Execution-verified data only Every training sample in OSS-Instruct exec-filter-50k has been independently run and verified to produce correct output. This eliminates a significant noise source that affects most open-source fine-tuning datasets: plausible-looking but incorrect code completions that silently degrade model performance on pass@1 metrics.
3. r=32 LoRA for speed without sacrificing quality At the 0.5B parameter scale, r=64 provides diminishing returns over r=32 while doubling the LoRA parameter count and training time. The r=32 configuration trains ~40% faster on the T4, allowing full training within a single Kaggle 9-hour session without checkpoint recovery.
4. Single Kaggle session design The entire pipeline — install → load → data → train → merge → upload → GGUF — is designed to complete within a single 9-hour Kaggle session. The 3-layer checkpoint recovery system (local → HuggingFace Hub → fresh start) handles session interruptions automatically when they occur.
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torchmodel = AutoModelForCausalLM.from_pretrained("Siddh07ETH/Atlas-Coder-2-0.5B",torch_dtype=torch.float16,device_map="auto",)tokenizer = AutoTokenizer.from_pretrained("Siddh07ETH/Atlas-Coder-2-0.5B")messages = [{"role": "system","content": "You are a helpful coding assistant."},{"role": "user","content": "Write a Python function to find all prime numbers up to n using the Sieve of Eratosthenes."}]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)with torch.no_grad():output = model.generate(**inputs,max_new_tokens=512,temperature=0.2,do_sample=True,top_p=0.9,repetition_penalty=1.1,)response = tokenizer.decode(output[0][inputs.input_ids.shape[1]:],skip_special_tokens=True)print(response)
python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigimport torchquant_config = BitsAndBytesConfig(load_in_4bit=True,bnb_4bit_quant_type="nf4",bnb_4bit_compute_dtype=torch.float16,)model = AutoModelForCausalLM.from_pretrained("Siddh07ETH/Atlas-Coder-2-0.5B",quantization_config=quant_config,device_map="auto",)tokenizer = AutoTokenizer.from_pretrained("Siddh07ETH/Atlas-Coder-2-0.5B")
GGUF quantizations for CPU inference are available at:
Runs at 40+ tokens/second on a laptop CPU. No GPU required.
| Setting | Value | Reason |
|---|---|---|
temperature | 0.1–0.3 | Low temperature for precise code generation |
top_p | 0.9 | Focused vocabulary sampling |
repetition_penalty | 1.1 | Prevents repeated patterns |
max_new_tokens | 256–512 | Sufficient for most single-function tasks |
If you are evaluating this model with EvalPlus, you must pass --model_type instruct:
bash
python -m evalplus.evaluate --model Siddh07ETH/Atlas-Coder-2-0.5B --dataset humaneval --backend hf --model_type instruct --greedy
Without --model_type instruct, EvalPlus sends raw function signatures without the ChatML wrapper. This causes the model to score near 0% — which is an evaluation configuration error, not a reflection of model quality. The model was trained exclusively on ChatML-formatted prompts and will not respond meaningfully to bare code signatures.
Atlas-Coder-2 is the second release in the Atlas-Coder series under Pluto AI. Each version refines the strategy based on lessons from the previous run.
| Version | Base | Strategy | Status |
|---|---|---|---|
| Atlas-Coder-0.5B (V1) | Qwen2.5-Coder-0.5B Base | 4-source 80K mixture, 3 epochs, r=64 | Published |
| Atlas-Coder-2-0.5B (V2) | Qwen2.5-Coder-0.5B-Instruct | 50K exec-verified, 1 epoch, r=32 | Flagship — this model |
| Model | Parameters | Description |
|---|---|---|
| Atlas-Coder-2-0.5B (this) | 494M | Flagship — exec-verified, instruct base |
| Atlas-Coder-0.5B (V1) | 494M | First generation, base model fine-tune |
| Pluto-Genesis-0.6B | 596M | General reasoning, math, and code |
Siddharth N.R Graduated B.Tech — AI & Data Science Pluto AI Research
bibtex
@misc{atlascoder2_2026,author = {Siddharth N.R.},title = {Atlas-Coder-2-0.5B: Execution-Verified QLoRA Fine-Tuning from an Instruct Base for Sub-1B Code Generation},year = {2026},publisher = {HuggingFace},url = {https://huggingface.co/Siddh07ETH/Atlas-Coder-2-0.5B}}
Apache 2.0 — see LICENSE. Base model Qwen2.5-Coder-0.5B-Instruct is also Apache 2.0.
| Sequence length |
| 1024 |
| 1024 |
| Response masking | ✅ | Standard LM loss |
| Final loss | 0.0294 | 0.0596 (healthy — not overfit) |
| Training time | ~42h 44m | ~9.5h |
| Fits in 1 Kaggle session | ❌ (needed resume) | ✅ |
| Training Steps | ~1,568 (resumed from checkpoint 250) |
| Final Training Loss | 0.0596 |
| Precision | FP16 (forced — T4 sm_75 does not support BF16) |
| Optimizer | AdamW 8-bit (Paged) |
| Learning Rate | 1e-4 (cosine schedule) |
| Warmup Steps | 100 |
| Effective Batch Size | 32 (2 × 16 grad accum) |
| Sequence Length | 1024 tokens |
| Hardware | Tesla T4 (16 GB VRAM) — Kaggle free tier |
| Training Time | ~9.5 hours (single session) |
| Framework | Transformers 4.52.4 + PEFT 0.17.0 + TRL 0.19.1 |
| Chat Template | ChatML (inherited from base instruct model) |
do_sample | True | Required when temperature < 1.0 |
