Base model
unsloth/Qwen3.6-35B-A3B
Adapter path from training
qwen36_35b_a3b_h200_official_bf16_lora64_24k_20260612/lora_adapter
Training / load settings
- max_seq_length:
24576
- dtype:
bfloat16
- adapter type: LoRA / PEFT
- trained with: Unsloth
- recommended merge target: clean BF16 base model
Load adapter with Unsloth
import torch
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="CyberNative-AI/Qwen36_35B_A3B_CyberNative_20260612_H200_bf16LoRA64_24k_LoRA",
max_seq_length=24576,
dtype=torch.bfloat16,
load_in_4bit=False,
token="hf_...",
)
Merge and save locally with Unsloth
model.save_pretrained_merged(
"merged_bf16",
tokenizer,
save_method="merged_16bit",
)
Merge and push with Unsloth
model.push_to_hub_merged(
"CyberNative-AI/YOUR_MERGED_REPO_NAME",
tokenizer,
save_method="merged_16bit",
token="hf_...",
)
Safer PEFT merge alternative
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained(
"unsloth/Qwen3.6-35B-A3B",
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
tok = AutoTokenizer.from_pretrained(
"unsloth/Qwen3.6-35B-A3B",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, "CyberNative-AI/Qwen36_35B_A3B_CyberNative_20260612_H200_bf16LoRA64_24k_LoRA")
model = model.merge_and_unload()
model.save_pretrained(
"merged_bf16",
safe_serialization=True,
max_shard_size="4GB",
)
tok.save_pretrained("merged_bf16")
Notes
If Unsloth merge prints warnings like:
MoE merge fallback
base weight is being written through
merged checkpoint will be missing this delta
do not trust that merged checkpoint. Use the PEFT merge route or update Unsloth/Unsloth Zoo and retry.