lananhyeuem

qwen3.5-27b-multilingual-rag-lora-n7

README

License: apache-2.0

Inference (vLLM)

python
from vllm import LLM, SamplingParams
from vllm.lora.request import LoRARequest

llm = LLM(model="Qwen/Qwen3.5-27B", enable_lora=True, max_lora_rank=128,
          trust_remote_code=True, dtype="bfloat16")
sp = SamplingParams(temperature=0.0, max_tokens=512)
out = llm.generate(prompts, sp,
                   lora_request=LoRARequest("a", 1, "<local path to this adapter>"))