CodeMate-Qwen-1.5B-8k 🚀
CodeMate-Qwen-1.5B-8k is a multi-turn, conversational coding assistant built on top of micymike/codemate-qwen-1.5B.
This iteration specifically resolves two critical limitations found in the previous model: it expands the core Context Window to 8,192 tokens using dynamic RoPE scaling, and it completely eliminates the prompt-repetition bug using targeted completion-only loss masking during Supervised Fine-Tuning (SFT).
Model Improvements 🛠️
- Extended Token Window: Upgraded from 2,048 tokens to 8,192 tokens, allowing it to read massive source code files and retain memory across deep, back-and-forth chat debug sessions.
- Prompt Echoing Fixed: Trained natively using Hugging Face TRL's
completion_only_loss=True. The model no longer mimics or repeats user instructions before outputting code.
- Bug-Fixing Specialization: Fine-tuned on multi-turn code repair paths (
iamtarun/code_instructions_120k_alpaca), teaching the model how to isolate programming syntax bugs, explain the root issues, and provide production-ready solutions without forgetting its baseline Python and ReactJS knowledge.
This model utilizes the standard ChatML prompt template structure. To prevent hallucination or parsing drops, structure your inference payloads like this:
<|im_start|>user[Your Python or React coding/debugging prompt goes here]<|im_end|><|im_start|>assistant
from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "micymike/codemate-qwen-1.5B-8k" model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")tokenizer = AutoTokenizer.from_pretrained(model_id) messages = [ {"role": "user", "content": "Fix the bug in this React code where the state hook updates infinitely."}] inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to("cuda")outputs = model.generate(inputs, max_new_tokens=1024, do_sample=True, temperature=0.3)print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Acknowledgements 🎓
Special thanks to the Unsloth AI framework for enabling memory-efficient 8k attention matrix mapping directly inside standard consumer GPU runtimes.