GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
sarimahsan101
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
sarimahsan101
Model Tree
lvwerra/gpt2-imdb
Input Modalities
Output Modalities
Supported Functionality
markdown
lvwerra/gpt2-imdb
A GPT-2 model already adapted to IMDB movie reviews.
markdown
lvwerra/distilbert-imdb
The reward for PPO is the probability assigned to the POSITIVE sentiment class.
stanfordnlp/imdbDuring preprocessing:
| Parameter | Value |
|---|---|
| Algorithm | PPO |
| TRL Version | 0.11.4 |
| Learning Rate | 1.41e-5 |
| Batch Size | 128 |
| Mini Batch Size | 16 |
| PPO Epochs | 4 |
| Adaptive KL | Enabled |
| Initial KL Coefficient | 0.2 |
| Target KL | 6.0 |
The policy was evaluated before and after PPO using identical prompts.
| Metric | Before PPO | After PPO |
|---|---|---|
| Mean Reward | 0.3746 | 0.9814 |
| Minimum Reward | 0.0037 | 0.8970 |
| Maximum Reward | 0.9941 | 0.9958 |
The model successfully learns to maximize the reward assigned by the sentiment classifier.
text
The movie started slow but
text
became a wonderful exercise in watching a film which makes you realise it's about love in a good way...
text
The director did a great job with
text
the movie. In spite of the technical limitations of this film it was a really fun film to watch...
This model is intended as an educational demonstration of RLHF.
Because the reward model measures only positive sentiment, PPO learns to maximize positive sentiment regardless of prompt consistency.
For example,
Prompt
text
I would not recommend this movie because
may become
text
it is entertaining, suspenseful and a fun ride!
Although this receives a high reward, it contradicts the prompt.
This behavior is a classic example of reward hacking (specification gaming) in reinforcement learning.
✅ Learning PPO
✅ Understanding RLHF
✅ Studying reward optimization
✅ Educational demonstrations
✅ TRL examples
bash
pip install transformers torch
python
from transformers import AutoTokenizer, AutoModelForCausalLMmodel_name = "sarimahsan101/ppo-gpt2-imdb"tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name)
python
import torchprompt = "The movie started slow but"inputs = tokenizer(prompt, return_tensors="pt")outputs = model.generate(**inputs,max_new_tokens=40,do_sample=True,top_p=0.9,temperature=0.8,pad_token_id=tokenizer.eos_token_id,)print(tokenizer.decode(outputs[0], skip_special_tokens=True))
python
from transformers import pipelinegenerator = pipeline("text-generation",model="sarimahsan101/ppo-gpt2-imdb")result = generator("Honestly, the acting in this film",max_new_tokens=40,do_sample=True,)print(result[0]["generated_text"])
This model was trained using:
markdown
transformers==4.46.3trl==0.11.4accelerate==0.34.2datasets==3.0.1peft==0.13.2tokenizers==0.20.3
If you use this model for educational or research purposes, please cite the repository.
bibtex
@misc{ppo_gpt2_imdb,author = {Syed Sarim Ahsan},title = {PPO-GPT2-IMDB},year = {2026},publisher = {Hugging Face},url = {https://huggingface.co/sarimahsan101/ppo-gpt2-imdb}}
Syed Sarim Ahsan
GitHub: https://github.com/sarimahsan101
Hugging Face: https://huggingface.co/sarimahsan101