Intended capabilities
The model is trained to assist with:
- Python bug fixing
- JavaScript and TypeScript debugging
- Production issue diagnosis
- Minimal unified-diff generation
- Regression-test generation
- API and backend debugging
- Repository maintenance
- Edge-case identification
- Production reliability improvements
Training data
The adapter was trained using curated examples from:
nebius/SWE-rebench-V2-PRs
The source dataset contains real-world GitHub pull requests and software-engineering tasks, including:
- Issue descriptions
- Repository metadata
- Base commit references
- Human-authored patches
- Test patches
- Python, JavaScript and TypeScript projects
Training examples were filtered to remove:
- Documentation-only changes
- Generated and minified files
- Lockfile-only changes
- Empty or malformed patches
- Unsupported programming languages
- Duplicate examples
- Examples whose complete answer exceeded the training sequence limit
Gold patches were preserved without cutting off their endings.
Language distribution
Table with columns: Language, Target examples| Language | Target examples |
|---|
| Python | 480 |
| TypeScript | 420 |
| JavaScript | 300 |
| Total | 1,200 |
Actual counts may differ slightly if the source stream did not contain enough qualifying examples for a specific language.
Training configuration
Table with columns: Setting, Value| Setting | Value |
|---|
| Base model | Qwen/Qwen3.5-2B |
| Training method | 4-bit QLoRA |
| Quantization | NF4 with double quantization |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| LoRA dropout | 0.05 |
| Maximum training length | 2,048 tokens |
| Epochs | 1 |
| Learning rate |
Training results
Table with columns: Metric, Result| Metric | Result |
|---|
| Training examples | REPLACE_WITH_TRAIN_SIZE |
| Evaluation examples | REPLACE_WITH_EVAL_SIZE |
| Training loss | REPLACE_WITH_TRAIN_LOSS |
| Evaluation loss | REPLACE_WITH_EVAL_LOSS |
| Epochs completed | 1 |
These loss values measure next-token prediction performance. They do not, by themselves, prove that generated patches compile or pass repository tests.
This repository contains a PEFT LoRA adapter, not standalone merged model weights.
The adapter must be loaded together with:
Installation
pip install -U "transformers>=5.2.0" peft accelerate bitsandbytes
Loading in 4-bit
import torch
from peft import PeftModel
from transformers import (
AutoTokenizer,
BitsAndBytesConfig,
Qwen3_5ForCausalLM,
)
BASE_MODEL_ID = "Qwen/Qwen3.5-2B"
ADAPTER_ID = "micymike/codemate-qwen3.5-2b-production-debugger"
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained(
ADAPTER_ID,
)
base_model = Qwen3_5ForCausalLM.from_pretrained(
BASE_MODEL_ID,
quantization_config=quantization_config,
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(
base_model,
ADAPTER_ID,
)
model.eval()
Inference example
import torch
messages = [
{
"role": "system",
"content": (
"You are CodeMate, a production software debugging "
"assistant. Diagnose the root cause, produce the smallest "
"correct patch, preserve unrelated behavior and add "
"regression tests when necessary."
),
},
{
"role": "user",
"content": """
Language: Python
Find and fix the production bug:
```python
from fastapi import FastAPI, HTTPException
app = FastAPI()
users = {1: {"name": "Michael"}}
@app.get("/users/{user_id}")
async def get_user(user_id: int):
user = users.get(user_id)
if user is None:
HTTPException(
status_code=404,
detail="User not found"
)
return {"id": user_id, "name": user["name"]}
Return the root cause, minimal unified diff and regression test.
""",
},
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(
prompt,
return_tensors="pt",
).to(model.device)
with torch.inference_mode():
output_ids = model.generate(
**inputs,
max_new_tokens=800,
do_sample=False,
repetition_penalty=1.05,
)
generated_ids = output_ids[
:,
inputs["input_ids"].shape[1]:
]
response = tokenizer.batch_decode(
generated_ids,
skip_special_tokens=True,
)[0]
print(response)
## Recommended prompt format
For repository tasks, provide as much relevant evidence as possible:
```text
Repository: owner/repository
Base commit: commit-sha
Language: Python | JavaScript | TypeScript
Issue:
Describe the observed behavior, expected behavior and available errors.
Relevant files:
<file path="src/example.py">
...
</file>
Failing tests:
...
Return:
1. Root-cause diagnosis
2. Minimal unified diff
3. Regression tests
4. Validation notes
Context length
The Qwen3.5 foundation supports long-context inference. However, this adapter’s supervised fine-tuning examples were limited to 2,048 tokens due to training hardware constraints.
Long-context support inherited from the base model does not guarantee equally strong debugging performance at every context length.
This release has not yet been comprehensively validated on:
- 16K repository contexts
- 32K repository contexts
- 64K repository contexts
- Full repository-scale agent workflows
The repository name and documentation should not claim verified 64K debugging performance until those evaluations are completed.
Evaluation status
Planned evaluations include:
- HumanEval+
- MBPP+
- SWE-bench-style patch generation
- Python repository debugging
- JavaScript and TypeScript repository debugging
- Unified-diff validity
- Regression-test generation
- 8K, 16K, 32K and 64K retrieval tests
- Executable patch validation
Evaluation benchmarks should remain separate from training data.
Limitations
CodeMate may:
- Generate patches that do not compile
- Misdiagnose incomplete issue reports
- Invent repository details when context is missing
- Produce syntactically valid but behaviorally incorrect changes
- Miss security, concurrency or compatibility issues
- Modify more code than necessary
- Generate incomplete regression tests
Always review and execute generated patches in an isolated development environment before production deployment.
Intended use
CodeMate is intended for:
- Software-engineering research
- Developer assistance
- Debugging experiments
- Patch-generation evaluation
- Educational coding workflows
It should not replace human code review, automated tests, security analysis or production deployment checks.
License
The adapter is released under the Apache 2.0 license.
Users are responsible for reviewing the licenses and terms associated with the base model, source dataset and any repositories represented in the training data.
Author
Created by micymike as part of the CodeMate model series.
