qwen3.6-27b-mythos5k-lora

Training Details

Dataset

Source: WithinUsAI/cluade_mythos_preview_5k_v2

Messages format with system/user/assistant roles. Applied via tokenizer.apply_chat_template().

GGUF (Weight-Diff)

A pre-converted LoRA GGUF is available for quick merging with llama.cpp:

bash
Copy code
# Merge with base model (CPU merge, requires ~503GB RAM for F16)
llama-export-lora --no-mmap \
    --model Qwen3.6-27B-Q8_0.gguf \
    --lora-scaled GGUF/qwen36-mythos5k-lora.gguf:1.0 \
    --output mythos5k-f16.gguf

# Quantize to smaller format
llama-quantize mythos5k-f16.gguf mythos5k-q4_k_m.gguf Q4_K_M

Inference with Transformers

python
Copy code
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.6-27B",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)
model = PeftModel.from_pretrained(
    base,
    "hotdogs/qwen3.6-27b-mythos5k-lora"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B", trust_remote_code=True)

messages = [
    {"role": "system", "content": "You are a master storyteller of myths and legends."},
    {"role": "user", "content": "Tell me about the creation of the world in a forgotten pantheon."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0]))

License

Apache 2.0

---

Created by UKA — 18yo coder & cybersecurity expert. June 18, 2026.

---

💖 Support / สนับสนุน

If you find this model useful, please consider supporting my work!
หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏

₿ Bitcoin — BTC:

markdown
Copy code
bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v

Thank you for your support! 🙏✨
ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗