GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
sageofai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
sageofai
Model Tree
Qwen/Qwen2.5-VL-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Our public leaderboard scores collapsed to BLEU = 0.0, ROUGE-1 = 0.22, METEOR = 0.17, while top systems sit at BLEU ≈ 0.38–0.46 and ROUGE-1 ≈ 0.84–0.89.
Diagnosing the previous submission_task1.py revealed the cause: the model was emitting full conversational sentences such as "The image shows a polyp in the sigmoid colon." while the Kvasir-VQA-x1 references are 1–3 word labels such as "polyp". BLEU is a geometric mean of n-gram precisions, so a single verbose token kills every n ≥ 2 and the score floors at zero. ROUGE / METEOR partially survive (recall-based), which matches the exact pattern we saw.
The fix is a three-pronged pipeline that turns the existing VLM into a short-label generator without retraining:
polyps → polyp, ulceration → ulcer, blood → bleeding, no abnormality → normal, erythematous mucosa → erythema, barretts esophagus → barrett's esophagus, …), question-type-aware label extraction, and length control (compress > 6 words, reject > 12 words).Decoding is also switched from sampling (T = 0.1, top_k = 20, top_p = 0.7, max_tokens = 64) to greedy (T = 0.0, top_k = 1, max_tokens = 32), since exact-match metrics punish sampling variance.
markdown
sageofai/Qwen25VL-MEDVQA-GI-S1-subtask1/├── submission_task1.py # the only entrypoint medvqa will run├── normalization.py # imported as a sibling module├── answer_bank.json # produced locally by build_answer_bank.py, committed to the repo├── requirements.txt├── README.md # this file└── (adapter weights — already there)
The QLoRA adapter weights are already in the repo; this fix-pack only adds / replaces the 5 small files above. answer_bank.json must be built locally and pushed alongside the script — the submission container has no network access to rebuild it.
bash
pip install -U datasetspython build_answer_bank.py
This writes answer_bank.json (~a few MB) containing:
answer_frequencies — global frequency of every observed training answerby_question_type — frequencies bucketed by yes_no / finding / count / color / …by_question_class — frequencies bucketed by the dataset's question_class fieldby_question_text — per-exact-question ranked answers (strongest signal when test questions repeat)all_unique_answers — flat set used for nearest-neighbour fallbackYou'll see the top 30 most frequent answers printed as a sanity check. They should look short and clinical (polyp, no, yes, normal, bleeding, ulcer, …).
bash
python test_normalization.py
Expected output: 34/34 passed. Covers yes/no edge cases, verbose model output, markdown wrapping, count words, color phrasing, canonical-map word boundaries, and category-filtered label extraction.
bash
python submission_task1.py
This loads the model + adapter, runs the validation split, writes predictions_1.json, and prints inline diagnostics. Watch for the FLAGS section at the bottom. It will warn if:
If any flag fires, stop and inspect before submitting.
bash
python local_validate.py predictions_1.json
Exits 0 if clean, 1 if flagged, 2 if the file is malformed. Useful if you want to inspect a predictions_1.json produced elsewhere (e.g. by the validation container) without re-running inference.
Push the five files (and the rebuilt answer_bank.json) to the HF repo root, then:
bash
pip install -U medvqamedvqa validate --competition=gi-2026 --task=1 --repo_id=sageofai/Qwen25VL-MEDVQA-GI-S1-subtask1medvqa validate_and_submit --competition=gi-2026 --task=1 --repo_id=sageofai/Qwen25VL-MEDVQA-GI-S1-subtask1
Deadline: 22 May 2026.
The "DO NOT EDIT" boundary in submission_task1.py is untouched:
SUBMISSION_INFO dict (team name, affiliations, contact) is unchangedHF_REPO_ID is unchangedevaluate block (bleu / rouge / meteor) is unchangedpredictions_1.json is unchanged (index, img_id, question, answer)medvqa validate_and_submit instructions printed at the end are unchangedAll edits live inside the two EDIT SECTION blocks the template explicitly allows.
| File | Purpose | Lines |
|---|---|---|
submission_task1.py | Drop-in replacement for the broken script. Greedy decoding, system + routed-user prompt, calls normalize_task1_answer on every output, prints inline diagnostics. | ~290 |
normalization.py | Core fix. Public API: normalize_task1_answer, get_question_type, nearest_answer, build_answer_bank. Self-contained, no heavy deps. | ~500 |
build_answer_bank.py |
submission_task1.py is structured so you can run four modes by toggling two flags near the top:
| Mode | USE_NORMALIZATION | USE_ANSWER_BANK |
|---|---|---|
| raw_model | False | False |
| normalized_model | True | False |
| normalized_with_answer_bank (default) | True |
Submit the strongest mode. Empirically normalized_with_answer_bank should win, since the bank rescues the ~5–15 % of outputs that escape canonicalization.
Peter Ojonugwa Ejiga — ojeji1@morgan.edu
Loads SimulaMet/Kvasir-VQA-x1 train split, calls build_answer_bank, writes answer_bank.json. |
| ~50 |
test_normalization.py | 34 torture cases covering every question type and every prefix / markdown / clause failure mode found during development. | ~120 |
local_validate.py | Standalone diagnostic over an existing predictions_1.json. Same FLAGS logic as the inline diagnostics. | ~90 |
requirements.txt | Pinned to the Kvasir-VQA-x1 reference setup (ms-swift==3.8.0, qwen_vl_utils==0.0.11). | — |
True| answer_bank_only_baseline | — | run separately via nearest_answer only |