🤖 Model Details
- Base Architecture: Qwen2
- Parameter Count: ~600M
- Languages: Korean, English
💻 Hardware & Compute
- Hardware: Trained on Kaggle Notebooks using 2x NVIDIA T4 GPUs (16GB VRAM each).
- Constraints: Navigating the memory constraints of a 600M parameter model on 16GB GPUs without advanced quantization required aggressive gradient accumulation and small batch sizes, leading to the decision to pivot to the highly efficient ~135M architecture for Bori-2 to allow for more robust pre-training experimentation.
⚠️ Limitations & Intended Use
This model was paused very early in its training lifecycle. It is significantly undertrained and exhibits poor coherence. It should not be used for text generation, fine-tuning, or deployment. Its primary value is as a historical artifact demonstrating the early stages of the Bori project's development.
