Training provenance
- Base-model revision:
cc594898137f460bfe9f0759e9844b3ce807cfb5.
- LLaMA-Factory LoRA SFT, rank 8, alpha 16; BF16 and DeepSpeed ZeRO-2.
- Datasets: Scene30k, SpatialSSRL_coldstart, and 3DThinker10k.
- Training examples: 39,451; validation examples: 4,384 (10% split).
- One complete epoch, 617 optimizer updates; effective batch size 64.
- Cluster: Trillium GPU (
grillium), node trig0033, 4 H100 GPUs.
- SLURM training job:
1019445; wall time: 20h 36m 27s.
- W&B training run.
Recorded evaluation
Table with columns: Step, Validation loss| Step | Validation loss |
|---|
| 200 | 1.010405 |
| 400 | 0.971601 |
| 600 | 0.961912 |
| 617 | 0.961902 |
Mean training loss: 1.037220. These metrics were measured with the base model plus LoRA adapter before merging. The merged export has not been separately benchmarked; BF16 merging can introduce small numerical differences. No downstream task accuracy is claimed.
Loading
Use this repository as model_name_or_path with a Qwen2.5-VL-compatible Transformers runtime. For LLaMA-Factory inference, use template: qwen2_vl and omit adapter_name_or_path. The model, processor, tokenizer, and chat template are packaged together.
Export
Exported offline through LLaMA-Factory llamafactory-cli export with infer_dtype=bfloat16, export_device=cpu, and safetensors shards of up to 5 GB. This repository contains inference weights, not optimizer state; preserve the original adapter checkpoint to resume the original training run.
The base model is licensed under Apache 2.0. Dataset terms continue to apply to their respective data. This is a research fine-tune; assess performance and suitability for the intended use.