GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
tinyrolls
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
tinyrolls
Model Tree
deepseek-ai/DeepSeek-R1-0528-Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
The paper's main table on the RoleVerse test set. Resp Avg: mean of five 1–5 response dimensions (GPT-5 judge). MCQ %: social-comprehension accuracy.
| Group | Model | Resp Avg | MCQ % |
|---|---|---|---|
| Untrained | Qwen2.5-32B-Instruct | 2.79 | 48.67 |
| R1-Distill-Qwen-32B | 2.38 | 53.28 | |
| Qwen3-30B-A3B-Instruct-2507 | 3.69 | 41.73 | |
| Qwen3-30B-A3B-Thinking-2507 | 3.40 | 52.27 | |
| gpt-oss-120b (low) | 3.15 | 67.30 | |
| gpt-oss-120b (medium) | 3.00 | 72.60 | |
| gpt-oss-120b (high) | 2.93 | 73.04 | |
| DeepSeek-R1-0528 | 4.12 | 70.14 | |
| GPT-5 (low; oracle, ref. only) | 4.35 | 81.88 | |
| Llama-3.1-8B-Instruct | base | 2.87 | 29.36 |
| + Only SFT | 3.35 | 19.07 | |
| + Only DPO | 3.42 | 32.39 | |
| + SFT & DPO | 3.81 | 17.05 | |
| Qwen3-8B Non-thinking | base | 3.16 | 33.96 |
| + Only SFT | 3.35 | 32.07 | |
| + Only DPO | 3.42 | 35.23 | |
| + SFT & DPO | 3.73 | 31.57 | |
| Qwen3-8B Thinking | base | 3.38 | 40.85 |
| + Only SFT | 3.38 | 37.75 | |
| + Only DPO | 3.60 | 43.12 | |
| + SFT & DPO | 3.61 | 34.60 | |
| R1-Distill-Qwen3-8B | base | 2.47 | 50.06 |
| + Only SFT | 2.15 | 31.50 | |
| + Only DPO | 3.07 | 49.62 | |
| + SFT & DPO | 3.07 | 28.85 |
The model speaks as role2; role1's utterances are user turns and role2's are assistant
turns. The system prompt is prompts/ResponseGeneration.yaml in the code repository.
python
from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "tinyrolls/DeepSeek-R1-0528-Qwen3-8B-RoleVerse-DPO"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto") dialogue = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": "<role1's utterance>"}, {"role": "assistant", "content": "<role2's utterance>"}, {"role": "user", "content": "<role1's utterance>"},]inputs = tok.apply_chat_template(dialogue, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(inputs, max_new_tokens=1024)utterance = tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True)Qwen3-8B checkpoints: pass enable_thinking=True to the Thinking and enable_thinking=False to
the NonThinking checkpoint in apply_chat_template. Thinking-mode checkpoints reply after the
</think> block.
Each checkpoint carries its base model's license as LICENSE: Llama 3.1 Community
License for Llama-3.1-8B-Instruct (Built with Llama; Llama 3.1 is licensed under the Llama 3.1
Community License, Copyright © Meta Platforms, Inc. All Rights Reserved), Apache-2.0 for Qwen3-8B,
MIT for DeepSeek-R1-0528-Qwen3-8B.
The paper is under review. Until it is published, please cite the dataset repository.
bibtex
@misc{roleverse2026, title = {Knows but Cannot Act: A Reasoning–Social Trade-off in Multilingual Role-Playing}, year = {2026}, url = {https://huggingface.co/datasets/tinyrolls/RoleVerse},}from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "tinyrolls/DeepSeek-R1-0528-Qwen3-8B-RoleVerse-DPO"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto") dialogue = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": "<role1's utterance>"}, {"role": "assistant", "content": "<role2's utterance>"}, {"role": "user", "content": "<role1's utterance>"},]inputs = tok.apply_chat_template(dialogue, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(inputs, max_new_tokens=1024)utterance = tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True)@misc{roleverse2026, title = {Knows but Cannot Act: A Reasoning–Social Trade-off in Multilingual Role-Playing}, year = {2026}, url = {https://huggingface.co/datasets/tinyrolls/RoleVerse},}