Model Details
Table | |
|---|
| Base model | unsloth/Qwen3-8B (Apache 2.0) |
| Method | QLoRA (4-bit), continued pretraining |
| Framework | Unsloth + TRL |
| Language | Bengali (bn) |
| Training data | LAYEK-143/BANGLA-BOOK |
| Status | v1 — first incremental round, more data to follow |
Intended Use
This checkpoint is intended as a base for further fine-tuning, not as a standalone chat assistant. Downstream uses include:
- Instruction/chat fine-tuning (SFT) on top of this checkpoint
- Research into Bengali language model adaptation
- A foundation for retrieval-augmented (RAG) applications requiring stronger native Bengali fluency
How to Use
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "LAYEK-143/Bangla-Qwen3-8B-CPT-v1",
max_seq_length = 2048,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model)
Limitations
- Not instruction-tuned. Expect continuation-style completions rather than direct question-answering.
- Corpus quality is mixed. Source material includes some content flagged as encoding-uncertain (legacy Bengali fonts); a cleaner v2 is planned as OCR correction progresses.
- Narrow domain skew. Training data leans toward formal/educational prose; conversational and informal registers are underrepresented.
License
Released under Apache 2.0, matching the base model. You are free to use, modify, and redistribute with attribution.
