The original training run loaded the base model from
<local-model-root>/Qwen2.5-7B-Instruct. The exact Hugging Face commit was not
recorded in the training artifacts and must be confirmed from that server-side
base-model directory before release. Do not claim a commit until it has been
verified.
Tokenizer: use the tokenizer from the same base-model revision.
LLaMA-Factory template: qwen.
LoRA configuration
- PEFT type: LoRA
- Rank: 8
- Alpha: 16
- Target modules:
q_proj, k_proj, v_proj, o_proj, gate_proj,
up_proj, and down_proj
Stage-I training
- Learning rate: 1e-4
- Scheduler: cosine
- Warmup ratio: 0.1
- Seed: 42
Loading
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = "Qwen/Qwen2.5-7B-Instruct"
adapter_dir = "path/to/adapter"
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype="auto")
model = PeftModel.from_pretrained(model, adapter_dir)
tokenizer = AutoTokenizer.from_pretrained(base_model)
To create a merged model, call model.merge_and_unload() and then
save_pretrained(). Distribution and use remain subject to the base model,
dataset, and original survey-source licenses.