yadavkapil7155

Qwen2.5-Coder-3B_QLora

README

License: apache-2.0

Overview

Large language models can generate code, explain algorithms, and assist with software development. However, fine-tuning can adapt their behavior to specific datasets and task requirements.

In this project, I fine-tuned an existing coding model using 4-bit QLoRA, trained it on coding instruction-response examples, saved the learned LoRA adapter, and tested inference and preliminary coding performance against the original base model.

Important: This project fine-tunes an existing pretrained model; it does not train a language model from scratch.

Project Highlights

Fine-tuned Qwen2.5-Coder-3B-Instruct using QLoRA.
Used 4-bit NF4 quantization to reduce GPU memory requirements.
Prepared a combined coding instruction dataset.
Trained on 45,000 examples with 2 epochs.
Used LoRA adapters instead of updating all base-model parameters.
Saved the final adapter and training checkpoints.
Ran inference using the base model with the trained adapter.
Compared the original and fine-tuned models on custom coding tests.

Model Architecture and Training

Table with columns: Parameter, Configuration
Parameter	Configuration
Base model	`Qwen/Qwen2.5-Coder-3B-Instruct`
Fine-tuning method	QLoRA
Quantization	4-bit NF4
Compute dtype	BF16 during training
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	`q_proj`, , , , , ,

QLoRA loads the base model in quantized form and trains additional low-rank adapter parameters. The original base weights remain frozen during training.

Dataset

The training dataset was constructed from two sources:

CodeAlpaca 20K — programming instructions and corresponding responses.
Magicoder-OSS-Instruct-75K — coding instruction-response examples.

Dataset Split

Table with columns: Split, Examples
Split	Examples
Training	45,000
Validation	2,994
Total prepared	47,994

The combined dataset includes programming instructions and examples covering multiple programming languages. Dataset metadata and language labels were retained where available.

Training Results

The production training run completed successfully after two epochs.

Table with columns: Metric, Result
Metric	Result
Training loss	0.4730
Validation loss	0.6010
Mean token accuracy	82.81%
Training duration	Approximately 14.85 hours
Final optimizer steps	5,626
Adapter size	Approximately 60 MB

These metrics describe the training run and its validation behavior. Mean token accuracy is not equivalent to the percentage of coding problems solved correctly.

Preliminary Evaluation

The original model and fine-tuned model were tested using the same prompts and predefined expected outputs on small, custom Python coding test suites.

Evaluation Results

Table with columns: Evaluation Set, Original Qwen, Fine-Tuned Qwen
Evaluation Set	Original Qwen	Fine-Tuned Qwen
Initial 5-problem pilot	12/15 (80.0%)	15/15 (100.0%)
Additional 20-problem pilot	52/60 (86.7%)	54/60 (90.0%)

The second evaluation showed a difference of 3.3 percentage points in favor of the fine-tuned model on this particular test suite.

These results are preliminary, not a definitive benchmark. The test suites are small and custom-built, and some problems overlap with earlier experiments. They do not establish general coding superiority or prove that the observed difference will generalize to unseen tasks.

A more rigorous evaluation should use a larger set of independent programming problems, validated test cases, and an isolated execution environment.

Inference

The trained adapter can be loaded alongside the original base model to generate code.

The inference process is:

Load the base Qwen model.
Load the tokenizer.
Attach the trained LoRA adapter.
Format a prompt using the model's chat template.
Generate a response.
Decode and inspect the generated output.

Example Prompt

text

Write a Python function to implement binary searchon a sorted list. Return the target index or -1if the target is not present.

The adapter contains the learned parameter updates; it is not a standalone replacement for the base model.

Project Structure

text

qwen-coder/├── data/│   ├── final/│   ├── train/│   └── validation/├── scripts/│   └── train_qlora.py├── checkpoints/│   └── qwen-coder-qlora/├── outputs/│   └── qwen-coder-qlora/│       ├── adapter_config.json│       ├── adapter_model.safetensors│       ├── eval_metrics.json│       └── training_metrics.json└── README.md

The exact contents may vary depending on which training checkpoints and generated artifacts are included in the repository.

Technologies Used

Python 3.11
PyTorch
Hugging Face Transformers
Hugging Face Datasets
PEFT
TRL
bitsandbytes
Google Colab
AWS EC2
Git and GitHub

Reproducing Inference

Install the required dependencies in a compatible Python environment:

bash

pip install torch transformers peft accelerate bitsandbytes

Load the base model and attach the adapter:

python

import torchfrom transformers import (    AutoTokenizer,    AutoModelForCausalLM,    BitsAndBytesConfig,)from peft import PeftModel BASE_MODEL = "Qwen/Qwen2.5-Coder-3B-Instruct"ADAPTER_PATH = "./outputs/qwen-coder-qlora" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) quant_config = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type="nf4",    bnb_4bit_compute_dtype=torch.float16,    bnb_4bit_use_double_quant=True,) base_model = AutoModelForCausalLM.from_pretrained(    BASE_MODEL,    quantization_config=quant_config,    device_map="auto",) model = PeftModel.from_pretrained(    base_model,    ADAPTER_PATH,) model.eval() messages = [    {        "role": "user",        "content": (            "Write a Python function to check whether "            "a string is a palindrome."        ),    }] inputs = tokenizer.apply_chat_template(    messages,    tokenize=True,    add_generation_prompt=True,    return_tensors="pt",).to(model.device) with torch.inference_mode():    outputs = model.generate(        inputs,        max_new_tokens=256,        do_sample=False,        pad_token_id=tokenizer.eos_token_id,    ) generated_tokens = outputs[0][inputs.shape[1]:] print(    tokenizer.decode(        generated_tokens,        skip_special_tokens=True,    ))

The code assumes the adapter files are available locally and that the installed libraries and hardware support the selected quantization configuration. BF16 was used for the original training run; FP16 is used here for broader inference compatibility.

Limitations

The project adapts an existing coding model rather than creating a model from scratch.
The adapter depends on the compatible base model.
Preliminary evaluation results are based on small, custom test suites.
Generated code may contain logical errors or security vulnerabilities and should be tested before use.
The current project does not claim a production-ready API, Docker deployment, or independently validated benchmark.

Future Improvements

Expand evaluation to a larger collection of unseen coding problems.
Test generated code against comprehensive test cases in an isolated sandbox.
Evaluate on established coding benchmarks where licensing and hardware permit.
Investigate failures and improve dataset quality.
Compare additional LoRA configurations and training hyperparameters.
Build a FastAPI inference service.
Add Docker packaging and deployment.
Publish reproducible evaluation results and example outputs.

Conclusion

This project demonstrates an end-to-end QLoRA fine-tuning workflow for an existing code-generation model: dataset preparation, quantized training, adapter saving, inference, and preliminary evaluation.

The initial experiments show promising results on selected custom test cases, while further independent evaluation is needed to determine the generality and significance of any performance improvement.

Acknowledgements

Qwen team for the Qwen2.5-Coder model.
Hugging Face for Transformers, Datasets, PEFT, and TRL.
The contributors to the CodeAlpaca and Magicoder datasets.

License and Dataset Attribution

Review the base model's license and the licenses and terms of both datasets before redistributing model artifacts or datasets. Preserve required attribution and comply with the applicable terms.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

yadavkapil7155

Model Tree

Base

Qwen/Qwen2.5-Coder-3B-Instruct

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Overview

Large language models can generate code, explain algorithms, and assist with software development. However, fine-tuning can adapt their behavior to specific datasets and task requirements.

Important: This project fine-tunes an existing pretrained model; it does not train a language model from scratch.

Project Highlights

Fine-tuned Qwen2.5-Coder-3B-Instruct using QLoRA.
Used 4-bit NF4 quantization to reduce GPU memory requirements.
Prepared a combined coding instruction dataset.
Trained on 45,000 examples with 2 epochs.
Used LoRA adapters instead of updating all base-model parameters.
Saved the final adapter and training checkpoints.
Ran inference using the base model with the trained adapter.
Compared the original and fine-tuned models on custom coding tests.

Model Architecture and Training

Table with columns: Parameter, Configuration
Parameter	Configuration
Base model	`Qwen/Qwen2.5-Coder-3B-Instruct`
Fine-tuning method	QLoRA
Quantization	4-bit NF4
Compute dtype	BF16 during training
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	`q_proj`, , , , , ,

QLoRA loads the base model in quantized form and trains additional low-rank adapter parameters. The original base weights remain frozen during training.

Dataset

The training dataset was constructed from two sources:

CodeAlpaca 20K — programming instructions and corresponding responses.
Magicoder-OSS-Instruct-75K — coding instruction-response examples.

Dataset Split

Table with columns: Split, Examples
Split	Examples
Training	45,000
Validation	2,994
Total prepared	47,994

The combined dataset includes programming instructions and examples covering multiple programming languages. Dataset metadata and language labels were retained where available.

Training Results

The production training run completed successfully after two epochs.

Table with columns: Metric, Result
Metric	Result
Training loss	0.4730
Validation loss	0.6010
Mean token accuracy	82.81%
Training duration	Approximately 14.85 hours
Final optimizer steps	5,626
Adapter size	Approximately 60 MB

These metrics describe the training run and its validation behavior. Mean token accuracy is not equivalent to the percentage of coding problems solved correctly.

Preliminary Evaluation

The original model and fine-tuned model were tested using the same prompts and predefined expected outputs on small, custom Python coding test suites.

Evaluation Results

Table with columns: Evaluation Set, Original Qwen, Fine-Tuned Qwen
Evaluation Set	Original Qwen	Fine-Tuned Qwen
Initial 5-problem pilot	12/15 (80.0%)	15/15 (100.0%)
Additional 20-problem pilot	52/60 (86.7%)	54/60 (90.0%)

The second evaluation showed a difference of 3.3 percentage points in favor of the fine-tuned model on this particular test suite.

A more rigorous evaluation should use a larger set of independent programming problems, validated test cases, and an isolated execution environment.

Inference

The trained adapter can be loaded alongside the original base model to generate code.

The inference process is:

Load the base Qwen model.
Load the tokenizer.
Attach the trained LoRA adapter.
Format a prompt using the model's chat template.
Generate a response.
Decode and inspect the generated output.

Example Prompt

text

Write a Python function to implement binary searchon a sorted list. Return the target index or -1if the target is not present.

Write a Python function to implement binary searchon a sorted list. Return the target index or -1if the target is not present.

The adapter contains the learned parameter updates; it is not a standalone replacement for the base model.

Project Structure

text

qwen-coder/├── data/│   ├── final/│   ├── train/│   └── validation/├── scripts/│   └── train_qlora.py├── checkpoints/│   └── qwen-coder-qlora/├── outputs/│   └── qwen-coder-qlora/│       ├── adapter_config.json│       ├── adapter_model.safetensors│       ├── eval_metrics.json│       └── training_metrics.json└── README.md

qwen-coder/├── data/│   ├── final/│   ├── train/│   └── validation/├── scripts/│   └── train_qlora.py├── checkpoints/│   └── qwen-coder-qlora/├── outputs/│   └── qwen-coder-qlora/│       ├── adapter_config.json│       ├── adapter_model.safetensors│       ├── eval_metrics.json│       └── training_metrics.json└── README.md

The exact contents may vary depending on which training checkpoints and generated artifacts are included in the repository.

Technologies Used

Python 3.11
PyTorch
Hugging Face Transformers
Hugging Face Datasets
PEFT
TRL
bitsandbytes
Google Colab
AWS EC2
Git and GitHub

Reproducing Inference

Install the required dependencies in a compatible Python environment:

bash

pip install torch transformers peft accelerate bitsandbytes

pip install torch transformers peft accelerate bitsandbytes

Load the base model and attach the adapter:

python

import torchfrom transformers import (    AutoTokenizer,    AutoModelForCausalLM,    BitsAndBytesConfig,)from peft import PeftModel BASE_MODEL = "Qwen/Qwen2.5-Coder-3B-Instruct"ADAPTER_PATH = "./outputs/qwen-coder-qlora" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) quant_config = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type="nf4",    bnb_4bit_compute_dtype=torch.float16,    bnb_4bit_use_double_quant=True,) base_model = AutoModelForCausalLM.from_pretrained(    BASE_MODEL,    quantization_config=quant_config,    device_map="auto",) model = PeftModel.from_pretrained(    base_model,    ADAPTER_PATH,) model.eval() messages = [    {        "role": "user",        "content": (            "Write a Python function to check whether "            "a string is a palindrome."        ),    }] inputs = tokenizer.apply_chat_template(    messages,    tokenize=True,    add_generation_prompt=True,    return_tensors="pt",).to(model.device) with torch.inference_mode():    outputs = model.generate(        inputs,        max_new_tokens=256,        do_sample=False,        pad_token_id=tokenizer.eos_token_id,    ) generated_tokens = outputs[0][inputs.shape[1]:] print(    tokenizer.decode(        generated_tokens,        skip_special_tokens=True,    ))

import torchfrom transformers import (    AutoTokenizer,    AutoModelForCausalLM,    BitsAndBytesConfig,)from peft import PeftModel BASE_MODEL = "Qwen/Qwen2.5-Coder-3B-Instruct"ADAPTER_PATH = "./outputs/qwen-coder-qlora" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) quant_config = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type="nf4",    bnb_4bit_compute_dtype=torch.float16,    bnb_4bit_use_double_quant=True,) base_model = AutoModelForCausalLM.from_pretrained(    BASE_MODEL,    quantization_config=quant_config,    device_map="auto",) model = PeftModel.from_pretrained(    base_model,    ADAPTER_PATH,) model.eval() messages = [    {        "role": "user",        "content": (            "Write a Python function to check whether "            "a string is a palindrome."        ),    }] inputs = tokenizer.apply_chat_template(    messages,    tokenize=True,    add_generation_prompt=True,    return_tensors="pt",).to(model.device) with torch.inference_mode():    outputs = model.generate(        inputs,        max_new_tokens=256,        do_sample=False,        pad_token_id=tokenizer.eos_token_id,    ) generated_tokens = outputs[0][inputs.shape[1]:] print(    tokenizer.decode(        generated_tokens,        skip_special_tokens=True,    ))

Limitations

The project adapts an existing coding model rather than creating a model from scratch.
The adapter depends on the compatible base model.
Preliminary evaluation results are based on small, custom test suites.
Generated code may contain logical errors or security vulnerabilities and should be tested before use.
The current project does not claim a production-ready API, Docker deployment, or independently validated benchmark.

Future Improvements

Expand evaluation to a larger collection of unseen coding problems.
Test generated code against comprehensive test cases in an isolated sandbox.
Evaluate on established coding benchmarks where licensing and hardware permit.
Investigate failures and improve dataset quality.
Compare additional LoRA configurations and training hyperparameters.
Build a FastAPI inference service.
Add Docker packaging and deployment.
Publish reproducible evaluation results and example outputs.

Conclusion

Acknowledgements

Qwen team for the Qwen2.5-Coder model.
Hugging Face for Transformers, Datasets, PEFT, and TRL.
The contributors to the CodeAlpaca and Magicoder datasets.

License and Dataset Attribution

Review the base model's license and the licenses and terms of both datasets before redistributing model artifacts or datasets. Preserve required attribution and comply with the applicable terms.