Project Motivation
Most small coding models can generate code, but they often struggle with structured debugging, practical explanations, and production-oriented guidance.
CodeMate was fine-tuned to respond like a senior engineering assistant by encouraging outputs such as:
- Clear implementation plans
- Self-contained code
- Root-cause debugging
- Practical explanations
- Frontend-focused React/Next.js/Tailwind examples
- Pythonic, readable, type-aware solutions
Base Model
Qwen/Qwen2.5-Coder-1.5B-Instruct
Training Method
The model was fine-tuned using:
- Unsloth
- QLoRA / 4-bit loading
- PEFT LoRA adapters
- SFTTrainer
- Cosine learning rate schedule
- AdamW 8-bit optimizer
LoRA configuration:
rank: 16lora_alpha: 32target_modules:- q_proj- k_proj- v_proj- o_proj- gate_proj- up_proj- down_proj
Usage
Install dependencies:
pip install unsloth transformers peft accelerate bitsandbytes
Load the LoRA adapter:
from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="micymike/codemate-qwen-lora", max_seq_length=2048, load_in_4bit=True,)
Example Output
Prompt:
Write a Python function to safely merge two dictionaries by summing matching numeric values.
Example response:
def safe_merge(dict1: dict, dict2: dict) -> dict: """Safely merges two dictionaries by summing values of keys that exist in both.""" merged_dict = {} for key in set(dict1.keys()).union(set(dict2.keys())): if key in dict1 and key in dict2: merged_dict[key] = dict1[key] + dict2[key] elif key in dict1: merged_dict[key] = dict1[key] else: merged_dict[key] = dict2[key] return merged_dict
Author
Built by Michael Moses / micymike as part of a practical AI engineering project focused on LLM fine-tuning, coding assistants, dataset curation, and deployment.