Training data
The fixed-budget training mixture has 5,801 rows: 5,201 unchanged v0.21
occurrences and 600 replacements. The replacements are the exact deterministic
300-row sample from AI Hub 71875 필수의료 의학지식 데이터 and 300-row sample
from AI Hub 71568 숫자연산 기계독해 데이터. Dataset landing pages:
AI Hub 71875
and AI Hub 71568.
No public benchmark data or benchmark answers were used for SFT.
Reproduction
- Base revision: 9b41bb2406472634d8812c0b8931fa40fa9a6c3a
- Objective: assistant-only causal-language-model cross entropy
- Epochs: 1
- Learning rate: 5e-5
- Optimizer: adamw_torch_fused
- Scheduler: linear; warmup steps 0
- Weight decay: 0.0
- LoRA: rank 16, alpha 32, dropout 0.05, bias none
- LoRA targets: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Per-device batch size: 1; gradient accumulation: 8
- Maximum sequence length: 2048
- Precision: BF16; packing disabled
- Random seed: 42
The full run used 5,801 records, built 5,793 tokenized examples, skipped 8
records during the original tokenizer contract check, and reached training
loss 0.3568975 after 725 optimizer steps.
Usage and limitations
Load this repository with transformers or standard vLLM. This is an
experimental model and may produce incorrect or unsafe answers; it must not
replace professional medical, financial, legal, or other expert advice.
License
The base model is distributed under Apache License 2.0. The applicable terms
of the AI Hub source data remain in force for use of the training data.