Architecture
MedJev-4B is built on Qwen3.5-4B with two changes and nothing else.
Language-side depth reduction. The first 23 of the 32 language layers are kept complete, layers
24 to 31 are deleted, and from layer 32 only the query and key projections are kept. The vision
encoder, the merger topology and the language forward are untouched, and every visual token the
merger produces is kept.
Residual decision head. A candidate is scored by the pretrained query--key geometry of the
backbone. The decision marker supplies the query, each option marker supplies a key, and the score is
their scaled dot product averaged over heads. A shared per-candidate MLP of hidden width 256 adds a
residual whose output layer is initialized to zero, so training starts from exactly the pretrained
matching score and learns only the correction.
Results
Accuracy (%) and mean end-to-end latency (ms) at batch size one on one H20.
Table with columns: Benchmark, Accuracy, Latency| Benchmark | Accuracy | Latency |
|---|
| HAM10000 | 68.39 | 73.3 |
| PneumoniaMNIST | 94.23 | 64.8 |
| MedThinkVQA | 51.94 | 250.7 |
| PMC-VQA | 61.68 | 65.0 |
| Average | 69.06 | 113.4 |
The latency average covers the three single-image benchmarks. MedJev-4B improves the average accuracy
of its own backbone by 7.93 points while answering 6.04 times faster.
Files
Table with columns: File, Content| File | Content |
|---|
adapter_model.safetensors | LoRA adapters, rank 16 and alpha 32, over 183 modules |
adapter_config.json | The PEFT configuration the adapters were trained with |
markers.pt | The three marker embeddings, shape (3, 2560) |
head.pt | The residual decision head, 1,966,593 parameters |
readout_norms.pt | The input, query and key norms of the readout, trained at full weight |
medjev_config.json | Depth, head, marker and budget record of the deployed model |
SHA256SUMS.json | Digest of every file above |
Trained tensors total 25,768,961 parameters. The backbone itself is not redistributed here, so the
base model is downloaded from Qwen/Qwen3.5-4B at load time.
Usage
git clone https://github.com/zfkarl/MedJev.git && cd MedJevpip install -e .hf download Karl28/MedJev-4B --local-dir checkpoints/medjev-4bbash scripts/evaluate.sh checkpoints/medjev-4b
medjev.evaluate merges the adapter into the backbone before timing, so the reported latency is not
inflated by adapter arithmetic. Each request is one forward pass, a prediction is a candidate index,
and it can never fail to match a candidate.
Training
One epoch over 174,360 examples from the training splits of the four benchmarks, 5,449 updates at a
global batch size of 32. The loss is a cross-entropy over the candidates of each example, weighted so
that every benchmark carries the same total weight and so that no diagnosis dominates inside a
classification benchmark.
Citation
Coming soon.
License
Apache 2.0, following the license of the Qwen3.5-4B backbone.