LaTexT
20251030-221219-sft-Qwen3-8B-ot3-1.2m-10k-converted-bs128-lr4e-5-epoch5-l18000
Available on FriendliAI
Dedicated EndpointsRun this model inference on single tenant GPU with unmatched speed and reliability at scale.
ContainerRun this model inference with full control and performance in your environment.
README
License: apache-2.0Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 4e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 128
- total_train_batch_size: 128
- total_eval_batch_size: 1024
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
Training results
Framework versions
- Transformers 4.51.1
- Pytorch 2.5.1+cu124
- Datasets 3.6.0
- Tokenizers 0.21.1