Model
- Name: Qwen3.5-27B
- Uses LoRA
- Effective batch size: 64
This repository contains the LoRA adapter only. To run inference you must
download the base model Qwen/Qwen3.5-27B separately and attach this
adapter to it — the adapter on its own cannot generate.
Approach
Every question is answered in context: a few nearest same-language question/answer
precedents are retrieved from a reference pool and placed in front of the
question, and the model is asked to answer by leaning on them. Training and
inference use the exact same layout, so one adapter is enough.
Intended use
Generating grounded answers to health questions in the supported languages, for
research and benchmark evaluation. Given a question and its retrieved precedents,
the model returns a single answer in the question's language.
Limitations
- Not medical advice. Outputs can be wrong or incomplete and must never guide
real clinical, diagnostic or treatment decisions.
- Quality is uneven across languages — stronger on higher-resource subsets,
weaker on the lowest-resource ones.
- The answer depends on the retrieved precedents; weak retrieval degrades it.
- Greedy decoding only, no calibrated confidence, no safety filtering.
Requirements
- 1× 80 GB GPU for inference
- Python packages:
vllm, sentence-transformers, pandas, numpy,
huggingface_hub
- A sentence-embedding retriever (
McGill-NLP/AfriE5-Large-instruct) to build the
precedent block
- The base backbone
Qwen/Qwen3.5-27B
Run inference
Open inference.ipynb and run it top to bottom. It will:
- Fetch this adapter and the retriever, and load the separately-downloaded
backbone.
- Build the precedent index from the reference pool.
- For each question, retrieve the top precedents, render the prompt, and
generate greedily.
- Write
submission.csv.
Set the environment variable LIMIT=16 first for a quick smoke test, or leave it
unset to score the whole test set. Point BASE_MODEL / ADAPTER_REPO at local
folders if you already have them.
Reproduce & verify
inference.ipynb regenerates submission.csv and runs structural checks
(columns, target consistency, id ordering, no empty rows). The reported score is
verified by uploading that file to the benchmark.
A note on variance. vLLM strongly affects both fine-tuning and inference —
re-running the same recipe through vLLM can move the score by up to ±0.004 on
both public and private. This has been observed repeatedly across many runs, so
the reported number should be read as a point within that band, not an exact
target.