Actual training
- Training word occurrences: 15,818 (not unique vocabulary or tokenizer tokens).
- One epoch, 15 optimizer updates; LoRA rank 8, alpha 16, query/value projections; 1,146,880 trainable parameters.
- FP32 on one NVIDIA TITAN X Pascal 12GB; learning rate 1e-5, 512-token chunks, batch 1 with accumulation 8.
- Verified native-reviewed instruction examples: 0. This run is prose continued pretraining, not instruction SFT.
Measured validation and limits
Table with columns: Diagnostic, Base, Adapted| Diagnostic | Base | Adapted |
|---|
| Token-weighted validation NLL | 4.218178 | 4.186697 |
| Perplexity | 67.909618 | 65.805046 |
Saved weights reloaded successfully and reproduced these validation values. These are heldout text-prediction diagnostics, not native fluency, accuracy, safety or conversational benchmarks. The original test partition was not used for these measurements.
Known failure: the recorded request for a Latin-script Balochi greeting did not reliably produce the requested script or natural language. The commercial adapter returned Arabic-script text; the research variant also showed repetition. No native reviewer has approved the samples. Do not infer native Balochi ability from the language tag.
Use
import torchfrom transformers import AutoTokenizerfrom peft import AutoPeftModelForCausalLMrepo = "shah-bakhsh/balcorpus-qwen3-0.6b-v1"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoPeftModelForCausalLM.from_pretrained(repo, torch_dtype=torch.float32)model.eval()
The adapter is released under Apache-2.0. Source text retains CC BY 4.0 attribution obligations; see SOURCE_CREDITS.md. No raw training corpus is bundled. Companion code: https://github.com/shah-bakhsh/balcorpus-balochi-assistant . Companion demo: https://huggingface.co/spaces/shah-bakhsh/balcorpus-balochi-demo .