GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
hrabalm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
hrabalm
Model Tree
utter-project/EuroLLM-9B-Instruct
Input Modalities
Output Modalities
Supported Functionality
We recommend using vLLM for inference, either directly or using the OpenAI-like server.
python
import vllmimport vllm.lora BASE_MODEL = "utter-project/EuroLLM-9B-Instruct"ADAPTER_MODEL = "hrabalm/CUNI-MH-v2-encs"llm = vllm.LLM( BASE_MODEL, enable_lora=True, max_lora_rank=32, enforce_eager=True, seed=42,)lora_request = vllm.lora.request.LoRARequest("adapter", 1, ADAPTER_MODEL) def format_prompt(src_lang, tgt_lang, src): return ( "Translate the following {src_lang} source text to {tgt_lang}:\n{src_lang}: {src}" ).format(src_lang=src_lang, tgt_lang=tgt_lang, src=src) sampling_params = vllm.SamplingParams( temperature=0, max_tokens=512, stop=["\n"],) messages = [ [{"role": "user", "content": format_prompt("English", "Czech", "Hello, world!")}]]outputs = llm.chat( messages, sampling_params=sampling_params, lora_request=lora_request,)print(outputs)print(outputs[0].outputs[0].text)bash
vllm serve \ utter-project/EuroLLM-9B-Instruct \ --dtype auto \ --seed 42 \ --enable-lora \ --host 0.0.0.0 \ --max-lora-rank 32 \ --max-num-seqs 256 \ --max-model-len 4096 \ --enforce-eager \ --lora-modules "default=hrabalm/CUNI-MH-v2-encs" # the model will be available under the name "default"Note that for the WMT25 submission, we split the input segments into segments of at most 256 input tokens on sentence boundaries. For this purpose, we used the sentence-splitter library.
import vllmimport vllm.lora BASE_MODEL = "utter-project/EuroLLM-9B-Instruct"ADAPTER_MODEL = "hrabalm/CUNI-MH-v2-encs"llm = vllm.LLM( BASE_MODEL, enable_lora=True, max_lora_rank=32, enforce_eager=True, seed=42,)lora_request = vllm.lora.request.LoRARequest("adapter", 1, ADAPTER_MODEL) def format_prompt(src_lang, tgt_lang, src): return ( "Translate the following {src_lang} source text to {tgt_lang}:\n{src_lang}: {src}" ).format(src_lang=src_lang, tgt_lang=tgt_lang, src=src) sampling_params = vllm.SamplingParams( temperature=0, max_tokens=512, stop=["\n"],) messages = [ [{"role": "user", "content": format_prompt("English", "Czech", "Hello, world!")}]]outputs = llm.chat( messages, sampling_params=sampling_params, lora_request=lora_request,)print(outputs)print(outputs[0].outputs[0].text)