Project links
Training configuration
- Base architecture: Qwen2 decoder-only Transformer, approximately 494M parameters
- Max sequence length: 256
- Steps: 300
- Batch size: 1
- Gradient accumulation: 2
- LoRA rank: 8
- LoRA alpha: 16
- LoRA dropout: 0.05
- LoRA targets:
q_proj, v_proj
- Trainable parameters: 540,672 (0.1093%)
The base weights are frozen. Causal language-modeling loss is computed only on
assistant tokens; system and user tokens are masked with -100.
Generated poetry may contain factual, tonal, metrical, or attribution errors.
Usage
from peft import AutoPeftModelForCausalLMfrom transformers import AutoTokenizer model_id = "YOUR_USERNAME/tang-song-poet-lora"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoPeftModelForCausalLM.from_pretrained(model_id)