GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ā
internlm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
internlm
Model Tree
Input Modalities
Output Modalities
Supported Functionality
We are working on even stronger base models and upgrading our training recipe ā stay tuned!
We are excited to introduce CapRL-3B, a lightweight 3B image captioner that achieves perception capabilities comparable to Qwen2.5-VL-72B.
This is the first study of applying Reinforcement Learning with Verifiable Rewards for the open-ended and subjective image captioning task. Unlike traditional Supervised Fine-Tuning, which can lead to models memorizing a limited set of annotated captions, our method allows the model to explore and generate a broader range of creative and general descriptions. CapRL is a new training paradigm featuring a decoupled two-stage pipeline. The initial stage uses LVLMs to generate rich and accurate captions. Subsequently, the second stage evaluates caption quality by using a vision-only LLM to perform the QA task. We also created a specific QA curation pipeline to ensure the quality of the questions and answers used for the second stage.
By employing the CapRL training framework, initializing with the Qwen2.5-VL-3B model, and using a carefully filtered 75K QA dataset as the training set, we obtained a highly capable captioner, CapRL-3B.
If you want to use CapRL-3B for captioning, you can directly follow the exact same inference approach as in Qwen2.5-VL-series.
We recommend using vLLM to speed up inference.
Run the command below to start an OpenAI-compatible API service:
bash
vllm serve "/PATH/CapRL-3B" \ --trust-remote-code \ --tensor-parallel-size=1 \ --pipeline-parallel-size=1 \ --gpu_memory_utilization=0.95 \ --served-model-name=caprl \ --port 8000 \ --host 0.0.0.0Then you can use the chat API as below: (see OpenAI API protocol document for more details):
python
import base64from openai import OpenAI# Set OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1"client = OpenAI( api_key=openai_api_key, base_url=openai_api_base,)image_path = "/path/to/local/image.png"with open(image_path, "rb") as f: encoded_image = base64.b64encode(f.read())encoded_image_text = encoded_image.decode("utf-8")base64_qwen = f"data:image;base64,{encoded_image_text}"chat_response = client.chat.completions.create( model="caprl", messages=[ {"role": "system", "content": "You are a helpful assistant."}, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": base64_qwen }, }, {"type": "text", "text": "What is the text in the illustrate?"}, ], }, ], temperature=1.0, max_tokens=max_tokens, top_p=1.0, extra_body={ "repetition_penalty": 1.0, },)print("Chat response:", chat_response)vllm serve "/PATH/CapRL-3B" \ --trust-remote-code \ --tensor-parallel-size=1 \ --pipeline-parallel-size=1 \ --gpu_memory_utilization=0.95 \ --served-model-name=caprl \ --port 8000 \ --host 0.0.0.0import base64from openai import OpenAI# Set OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1"client = OpenAI( api_key=openai_api_key, base_url=openai_api_base,)image_path = "/path/to/local/image.png"with open(image_path, "rb") as f: encoded_image = base64.b64encode(f.read())encoded_image_text = encoded_image.decode("utf-8")base64_qwen = f"data:image;base64,{encoded_image_text}"chat_response = client.chat.completions.create( model="caprl", messages=[ {"role": "system", "content": "You are a helpful assistant."}, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": base64_qwen }, }, {"type": "text", "text": "What is the text in the illustrate?"}, ], }, ], temperature=1.0, max_tokens=max_tokens, top_p=1.0, extra_body={ "repetition_penalty": 1.0, },)print("Chat response:", chat_response)