Stages
Table with columns: #, stage, where, trained on, r / alpha| # | stage | where | trained on | r / alpha |
|---|
| 1 | human-ko-sft | ThakiCloud/Qwen3.8-27B-Human-KO-LoRA | see that repo | see that repo |
| 2 | enterprise-sft | repo root | ThakiCloud/Qwen3.8-27B-Human-KO | 8 / 16 |
Apply the stages in this order. Each stage was trained on the model produced by the previous one.
How to use
import torchfrom transformers import AutoModelForCausalLMfrom peft import PeftModelĀ model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", torch_dtype=torch.bfloat16)model = PeftModel.from_pretrained(model, "ThakiCloud/Qwen3.8-27B-Human-KO-LoRA").merge_and_unload() # stage 1model = PeftModel.from_pretrained(model, "ThakiCloud/Qwen3.8-27B-Human-KO-Enterprise-v0.1-LoRA").merge_and_unload() # stage 2: enterprise-sft
How we checked these are the right adapters
For 4 randomly chosen LoRA target tensors we compared the published merged weights minus ThakiCloud/Qwen3.8-27B-Human-KO with the adapter deltas (scale x B x A).
The median cosine is 0.924. Rounding the merged weights to bf16 keeps it below 1; an adapter that was not used scores about 0.
Notes
adapter_config.json names the public model each stage was trained on. Training used a local copy of the same weights, and the adapter weights are unchanged.
- Stages after the Human-KO SFT were trained on the Human-KO merge before its lm_head row prune. The LoRA does not touch lm_head, so on the public Human-KO these adapters reproduce every LoRA-targeted tensor of the published model; the published model keeps the unpruned lm_head, so use the merged repo for bit-exact weights.
License
Same as ThakiCloud/Qwen3.8-27B-Human-KO-Enterprise-v0.1: apache-2.0.