Model Details
- Base model: Qwen/Qwen2.5-7B-Instruct
- Fine-tuning method: QLoRA
- Quantization: 4-bit NF4
- LoRA rank: 16
- LoRA alpha: 32
- LoRA dropout: 0.05
- Trainable parameters: ~40.4M
- Trainable parameter ratio: ~0.53%
- Training epochs: 2
- Dataset examples:
- Train: 12,711
- Validation: 1,552
- Test: 1,582
Training Dataset
The training dataset was generated from structured astronomy observations obtained from the NASA Exoplanet Archive.
The dataset contains four primary tasks:
- Observation interpretation
- Orbital reasoning
- Planet property reasoning
- Stellar dynamics
The dataset includes structured metadata such as planet name, host star, discovery method, discovery year, orbital period, stellar mass, stellar radius, and other available astronomical properties.
Training
The model was fine-tuned using 4-bit QLoRA with the following configuration:
r = 16
lora_alpha = 32
lora_dropout = 0.05
target_modules:
q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
quantization:
4-bit NF4
double quantization: enabled
compute dtype: float16