Mpairwe49

retinalai-narrator-135m

README

License: apache-2.0

What is different about it

Prose only. It emits the narrative as plain text. The structured triage decision is served separately by a 3 KB linear head, so this model has no JSON contract to break. An earlier JSON-emitting version of the same base model failed to parse on 100% of cases once 4-bit quantized; prose-only generation is 1.000 at every precision.
Pruned vocabulary, 49,152 -> 920 tokens. Embedding rows are the original tokenizer ids listed in keep_ids.json, in order. The tokenizer is unmodified; callers map ids old->new before the model and new->old before decoding. Tokens outside the kept set are re-encoded byte-wise rather than dropped. The vocabulary is seeded from all 45 disease names the upstream classifier can emit, and all 45 are verified to survive the round trip.

Use src/narrator/compact.py from the OptiscanAI repository — the id remap is required, so AutoModelForCausalLM alone will produce garbage.

Evaluation

24 held-out cases, scored against the teacher's own narrative for the same case:

Table with columns: precision, size, generation rate, omission, misquoted probabilities, acuity divergence
precision	size	generation rate	omission	misquoted probabilities	acuity divergence
bf16	213.5 MB	1.000	0.000	0.000	0.250
int8	107.3 MB	—	—	—	— (37.5 s/case, impractical)
nf4	54.2 MB	1.000	0.417	0.167	0.583

bf16 is the only precision recommended. 4-bit reaches 54 MB but drops findings the teacher reported in 42% of cases while remaining fluent, which makes the omissions hard to notice.

"Acuity divergence" counts narratives using urgency language the teacher did not use for that same case. It is a broad screening signal, not a hallucination count.

Limitations

Not a medical device. Not for clinical use. No regulatory clearance, no clinical validation, no prospective study.
Evaluated on 24 held-out cases from a single RFMiD-derived sample. That is enough to distinguish "works" from "broken" and little else.
No clinician review of generated text has been performed.
Trained on 56 examples. It reproduces the teacher's register on cases that look like its training distribution and has not been probed outside it.
No EMERGENCY case appeared anywhere in the source data, so emergency phrasing is untested. Escalation is handled deterministically upstream, not here.
Callers must append their own AI-disclosure statement; the teacher traces did not contain one, so the model does not produce it.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Mpairwe49

Model Tree

Base

HuggingFaceTB/SmolLM2-135M-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities