NEXS-qwen3-32b-medical-openmedzoo-lora API & Inference Endpoint

Sanitization applied

The raw mergekit extraction included full-rank modules_to_save tensors (embed_tokens, lm_head, and norm layers) that vLLM's LoRA runtime does not support. This upload contains only the pure low-rank lora_A/lora_B weights (448 pairs: 64 layers x q/k/v/o/gate/up/down projections), with modules_to_save: null in adapter_config.json. No resize_token_embeddings() call is needed to load this adapter.

Serving with vLLM

bash
python -m vllm.entrypoints.openai.api_server \
    --model Qwen/Qwen3-32B \
    --enable-lora \
    --lora-modules medical_openmedzoo=anjohn0077/NEXS-qwen3-32b-medical-openmedzoo-lora \
    --port 8000 \
    --max-lora-rank 128 \
    --gpu-memory-utilization 0.85

Evaluation (medqa_4options)

Table with columns: Variant, Accuracy
Variant	Accuracy
Base model	0.7494
This LoRA on base (via vLLM)	0.7604
Original full fine-tune	0.7596

Evaluated with lm-evaluation-harness against a local vLLM OpenAI-compatible endpoint:

bash
lm_eval --model local-completions \
    --model_args model=medical_openmedzoo,base_url=http://localhost:8000/v1/completions,tokenizer=Qwen/Qwen3-32B,num_concurrent=10 \
    --tasks medqa_4options \
    --output_path results/vllm_medical_openmedzoo

Sanitization applied

Serving with vLLM

bash
python -m vllm.entrypoints.openai.api_server \
    --model Qwen/Qwen3-32B \
    --enable-lora \
    --lora-modules medical_openmedzoo=anjohn0077/NEXS-qwen3-32b-medical-openmedzoo-lora \
    --port 8000 \
    --max-lora-rank 128 \
    --gpu-memory-utilization 0.85

Evaluation (medqa_4options)

Table with columns: Variant, Accuracy
Variant	Accuracy
Base model	0.7494
This LoRA on base (via vLLM)	0.7604
Original full fine-tune	0.7596

Evaluated with lm-evaluation-harness against a local vLLM OpenAI-compatible endpoint:

bash
lm_eval --model local-completions \
    --model_args model=medical_openmedzoo,base_url=http://localhost:8000/v1/completions,tokenizer=Qwen/Qwen3-32B,num_concurrent=10 \
    --tasks medqa_4options \
    --output_path results/vllm_medical_openmedzoo

NEXS-qwen3-32b-medical-openmedzoo-lora

Get help setting up a custom Dedicated Endpoints.

README

Sanitization applied

Serving with vLLM

Evaluation (medqa_4options)

Explore FriendliAI today

README

Sanitization applied

Serving with vLLM

Evaluation (medqa_4options)