Provenance
- Base model:
openbmb/MiniCPM5-2B
- Base revision:
12a3808a956f869c767195e9266b59c4d21d92e2
- Training method: BF16 LoRA, not QLoRA
- Trainer: Unsloth native loading with TRL
SFTTrainer
- Hardware: 1 x NVIDIA A40 48 GB
- Maximum sequence length: 8192
- Trainable parameters observed: 100,466,688
- Total parameters observed: 2,617,223,168
- Trainable fraction: 3.8387%
Dataset and tokens
- Training examples: 95347
- Eligible selected examples: 95347
- Quarantined examples excluded: 1153
- Prompt tokens: 19578751
- Supervised assistant tokens: 56063413
- Unpadded dataset tokens: 75642164
- Trainer-reported input tokens processed: 107296408
Only assistant/TikZ tokens contributed to loss. Prompt tokens were masked and
overlength examples were quarantined rather than truncated.
Optimization
- Epochs completed: 1.000000
- Effective batch size: 16
- Per-device batch size: 2
- Gradient accumulation: 8
- Learning rate: 0.0001
- Scheduler: cosine
- LoRA rank: 64
- LoRA alpha: 64
- LoRA dropout: 0.0
Results
- Overall gate passed: True
- Training completed: True
- Full epoch completed: True
- Artifact complete: True
- Validation loss: 0.449503
- Trainer-reported train loss: 0.061229
- First logged loss: 0.589100
- Final logged loss: 0.572400
- Minimum logged loss: 0.363200
- Mean logged loss: 0.537604
- Trainer-reported input tokens: 107296408
- Total FLOPs: 1.512773958542721e+18
- Accumulated Slurm allocation time: 10h 9m 38s
- Training allocations: 6
Loss is token-level cross-entropy against one reference TikZ implementation;
it does not directly measure compilation or rendered-image similarity.
Slurm allocations
Table with columns: Job ID, Name, State, Elapsed, Node| Job ID | Name | State | Elapsed | Node |
|---|
| 4317286 | minicpm-native-full | TIMEOUT | 01:00:03 | a0429 |
| 4321521 | minicpm-native-full-r2 | TIMEOUT | 02:00:25 | a0429 |
| 4322440 | minicpm-native-full-r3 | TIMEOUT | 02:00:17 | a0429 |
| 4324001 | minicpm-native-full-r4 | TIMEOUT | 02:00:20 | a0429 |
| 4326352 | minicpm-native-full-r2 | TIMEOUT | 02:00:22 | a0429 |
| 4327247 | minicpm-native-final | COMPLETED | 01:08:11 | a0226 |
Loading
The standard Transformers PEFT integration can load the adapter using its
recorded base-model reference:
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "Praha-Labs/MiniCPM5-2B-TikZ-LoRA"tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( repo, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)
Explicit PEFT loading against the pinned upstream base is also supported:
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel repo = "Praha-Labs/MiniCPM5-2B-TikZ-LoRA"base = AutoModelForCausalLM.from_pretrained( "openbmb/MiniCPM5-2B", revision="12a3808a956f869c767195e9266b59c4d21d92e2", torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,)tokenizer = AutoTokenizer.from_pretrained(repo)model = PeftModel.from_pretrained(base, repo)
Included evidence
training-metrics.json, resolved-config.json, training-config.yaml,
native-run.json, token-report.json, training-jobs.tsv,
environment.lock, source-commit.txt, BASE_MODEL_CARD.md, and
SHA256SUMS document the run and its provenance.
Limitations
Compilation rate, rendered-image similarity, and human evaluation should be
measured before deployment because multiple valid TikZ programs can render the
same diagram.