Training
Stage 1 -- Supervised Fine-Tuning (SFT)
How to use
Load adapters with Unsloth (recommended, 2x faster inference)
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
model_name = "ihumaunkabir/qwen3_bangla_lora",
max_seq_length = 2048,
load_in_4bit = True,
)
FastModel.for_inference(model)
alpaca_prompt = """নিচে একটি নির্দেশনা দেওয়া আছে, যা একটি কাজের বর্ণনা দেয়। অনুরোধটি যথাযথভাবে সম্পূর্ণ করে একটি উত্তর লিখুন।
### নির্দেশনা:
{}
### উত্তর:
{}"""
inputs = tokenizer([alpaca_prompt.format("বাংলাদেশের রাজধানীর নাম লেখো।", "")], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=128, use_cache=True)
print(tokenizer.batch_decode(outputs))
Note: this uses Unsloth's FastModel API (not FastLanguageModel). FastModel is the unified
entry point for newer model families including Qwen3.
Load with PEFT (no Unsloth)
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model = AutoPeftModelForCausalLM.from_pretrained(
"ihumaunkabir/qwen3_bangla_lora",
load_in_4bit = True,
)
tokenizer = AutoTokenizer.from_pretrained("ihumaunkabir/qwen3_bangla_lora")
Limitations
- Smoke-test training volume (
max_steps=120) -- ~1,920 samples. Increase for production.
- No safety alignment. No RLHF, DPO, or red-teaming.
- QLoRA 4-bit during training can slightly degrade quality vs. bf16.
- Requires the base model to be loaded separately -- this is an adapter, not a standalone model.
- Alpaca prompt format required. The model will produce poor output if prompted with a
different template (e.g. ChatML).
Citation
If you use these adapters, please cite both this repo and the alpaca-gpt4-bangla dataset,
plus the base Qwen3 model.
This repo (LoRA adapters)
@misc{qwen3-bangla-lora,
author = {ihumaunkabir},
title = {qwen3-bangla: Bangla SFT LoRA adapters for Qwen3-4B-Base},
year = {2026},
url = {https://huggingface.co/ihumaunkabir/qwen3_bangla_lora},
note = {SFT on alpaca-gpt4-bangla, trained with Unsloth}
}
SFT dataset -- alpaca-gpt4-bangla
@misc{alpaca-gpt4-bangla,
author = {ihumaunkabir},
title = {alpaca-gpt4-bangla: A Bangla instruction-following dataset},
year = {2026},
url = {https://huggingface.co/datasets/ihumaunkabir/alpaca-gpt4-bangla},
note = {Machine translation (Korean -> Bangla) of FreedomIntelligence/alpaca-gpt4-korean}
}
Base model
@misc{qwen3,
author = {Qwen Team},
title = {Qwen3-4B-Base},
year = {2025},
url = {https://huggingface.co/Qwen/Qwen3-4B-Base}
}
