GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
princeton-nlp
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
princeton-nlp
Model Tree
google/gemma-2-9b-it
Input Modalities
Output Modalities
Supported Functionality
We fine-tuned google/gemma-2-9b-it on princeton-nlp/gemma2-ultrafeedback-armorm with the SimPO objective.
markdown
import torchfrom transformers import pipelinemodel_id = "princeton-nlp/gemma-2-9b-it-SimPO"generator = pipeline("text-generation",model=model_id,model_kwargs={"torch_dtype": torch.bfloat16},device="cuda",)outputs = generator([{"role": "user", "content": "What's the difference between llamas and alpacas?"}],do_sample=False,eos_token_id=[generator.tokenizer.convert_tokens_to_ids("<end_of_turn>"), generator.tokenizer.eos_token_id],max_new_tokens=200)print(outputs[0]['generated_text'])
We use princeton-nlp/gemma2-ultrafeedback-armorm as the preference optimization dataset.
The hyperparameters used can be found in the training script.
Fine-tuning the google/gemma-2-9b-it on princeton-nlp/gemma2-ultrafeedback-armorm takes around 100 mins to finish on 8xH100 GPUs.
| models | AE2 LC | AE2 WR | AE2 Length | AH | AH Length | GSM | GSM Length | MMLU | MMLU Length |
|---|---|---|---|---|---|---|---|---|---|
| google/gemma-2-9b-it | 51.1 | 38.1 | 1571 | 40.8 | 545 | 87.4 | 395 | 72.7 |
The model architecture is based on google/gemma-2-9b-it. We use the SimPO training objective proposed in our preprint.
We used 8xH100 GPUs for model training.
Training was done using the alignment-handbook library.
gemma model:
markdown
@article{gemma_2024,title={Gemma},url={https://www.kaggle.com/m/3301},DOI={10.34740/KAGGLE/M/3301},publisher={Kaggle},author={Gemma Team},year={2024}}
SimPO paper:
markdown
@article{meng2024simpo,title={{SimPO}: Simple preference optimization with a reference-free reward},author={Meng, Yu and Xia, Mengzhou and Chen, Danqi},journal={arXiv preprint arXiv:2405.14734},year={2024}}
UltraFeedback paper:
markdown
@article{cui2023ultrafeedback,title={{UltraFeedback}: Boosting language models with high-quality feedback},author={Cui, Ganqu and Yuan, Lifan and Ding, Ning and Yao, Guanming and Zhu, Wei and Ni, Yuan and Xie, Guotong and Liu, Zhiyuan and Sun, Maosong},journal={arXiv preprint arXiv:2310.01377},year={2023}}
ArmoRM paper:
markdown
@article{wang2024interpretable,title={Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts},author={Wang, Haoxiang and Xiong, Wei and Xie, Tengyang and Zhao, Han and Zhang, Tong},journal={arXiv preprint arXiv:2406.12845},year={2024}}
| 515 |
| princeton-nlp/gemma-2-9b-it-DPO | 67.8 | 65.4 | 2016 | 58.9 | 717 | 88.5 | 392 | 72.2 | 624 |
| princeton-nlp/gemma-2-9b-it-SimPO | 72.4 | 65.9 | 1833 | 59.1 | 693 | 88.0 | 341 | 72.2 | 441 |