Qwen36_35B_A3B_CyberNative_20260612_H200_bf16LoRA64_24k_LoRA

Base model

unsloth/Qwen3.6-35B-A3B

Adapter path from training

qwen36_35b_a3b_h200_official_bf16_lora64_24k_20260612/lora_adapter

Training / load settings

• max_seq_length: 24576
• dtype: bfloat16
• adapter type: LoRA / PEFT
• trained with: Unsloth
• recommended merge target: clean BF16 base model

Load adapter with Unsloth

python
Copy code
import torch
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="CyberNative-AI/Qwen36_35B_A3B_CyberNative_20260612_H200_bf16LoRA64_24k_LoRA",
    max_seq_length=24576,
    dtype=torch.bfloat16,
    load_in_4bit=False,
    token="hf_...",
)

Merge and save locally with Unsloth

python
Copy code
model.save_pretrained_merged(
    "merged_bf16",
    tokenizer,
    save_method="merged_16bit",
)

Merge and push with Unsloth

python
Copy code
model.push_to_hub_merged(
    "CyberNative-AI/YOUR_MERGED_REPO_NAME",
    tokenizer,
    save_method="merged_16bit",
    token="hf_...",
)

Safer PEFT merge alternative

python
Copy code
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "unsloth/Qwen3.6-35B-A3B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

tok = AutoTokenizer.from_pretrained(
    "unsloth/Qwen3.6-35B-A3B",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(base, "CyberNative-AI/Qwen36_35B_A3B_CyberNative_20260612_H200_bf16LoRA64_24k_LoRA")
model = model.merge_and_unload()

model.save_pretrained(
    "merged_bf16",
    safe_serialization=True,
    max_shard_size="4GB",
)

tok.save_pretrained("merged_bf16")

Notes

If Unsloth merge prints warnings like:

• MoE merge fallback
• base weight is being written through
• merged checkpoint will be missing this delta

do not trust that merged checkpoint. Use the PEFT merge route or update Unsloth/Unsloth Zoo and retry.