nikitastheo

babylm-ita-ell-interleaved-dummy

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Container

Run this model inference with full control and performance in your environment.

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 16
training_steps: 128

Table
Training Loss	Epoch	Step	Validation Loss	Accuracy
10.8312	5.0	5	10.8354	0.0018
10.8147	10.0	10	10.8292	0.0018
10.7816	15.0	15	10.8142	0.0016
10.7263	20.0	20	10.7806	0.0012
10.6647	25.0	25	10.7343	0.0006
10.6091	30.0	30	10.6908	0.0002
10.5569	35.0	35	10.6538	0.0008
10.5058	40.0	40	10.6205	0.0010
10.4558	45.0	45	10.5892	0.0016
10.409	50.0	50	10.5586	0.0022
10.3654	55.0	55	10.5285	0.0025
10.3244	60.0	60	10.5007	0.0031
10.2865	65.0	65	10.4756	0.0041
10.2518	70.0	70	10.4524	0.0051
10.2201	75.0	75	10.4312	0.0057
10.1915	80.0	80	10.4121	0.0059
10.1659	85.0	85	10.3951	0.0059
10.1432	90.0	90	10.3802	0.0061
10.1235	95.0	95	10.3673	0.0061
10.1065	100.0	100	10.3563	0.0063
10.0923	105.0	105	10.3472	0.0063
10.0809	110.0	110	10.3399	0.0063
10.0721	115.0	115	10.3343	0.0061
10.0659	120.0	120	10.3305	0.0061
10.0623	125.0	125	10.3285	0.0061