clinical-jev-qwen3.5-2b-mtsamples API & Inference Endpoint

In one minute

You give the model a clinical note and a multiple-choice question, for example "What type of document is this?". It answers with a probability for every option, such as operative note 99%, consult 0.4%, ….

It does this in one step, without writing any text. That makes it:

Accurate: 95% on document type, 91% on specialty and 97.5% on "was a procedure performed?". The same model scored 65%, 75% and 84% before training.
Fast: 1.8× faster than asking a normal chatbot-style model to write its answer.
Safe to automate: the probabilities can be trusted. 99% of notes can be sorted by document type automatically while staying under 5% error, against 44% before training. The rest go to a person.
Always well-formed: it can only pick one of the options you give it, so the answer never needs parsing and can never be malformed.

Reading the answer is faster; training makes it right

The idea: "Jev-style" decisions instead of chatting

Most AI systems answer a question by writing text ("The answer is B…"), which your code then has to parse. That is slow, and the answer can come back in the wrong format.

This model follows the Jev / "System One" idea:

text

                     normal chatbot                      this model (Jev-style)note + question ──►  writes "B" token by token  ──►      reads the model's preference for A, B, C… in ONE step                     then code parses the text            ──► {A: 2%, B: 95%, C: 3%}  (nothing to parse)

Technically, we take Qwen/Qwen3.5-2B-Base and add a small trained add-on (a LoRA adapter: 16.8M parameters, under 1% of the model). At the end of the prompt (### Answer:), we read how strongly the model "wants" to say each option letter and turn that into probabilities.

The three questions it answers

Table with columns: Question, Kind, Possible answers
Question	Kind	Possible answers
`doc_type`: What type of document is this?	pick one of 7	operative/procedure note · consult or history & physical · radiology/diagnostic report · progress/SOAP note · discharge summary · ER report · other (office note, letter, etc.)
`specialty`: Which specialty does it belong to?	pick one of 16	cardiology/pulmonary · orthopedics · general medicine · GI · neurology · urology · OB/GYN · ENT · neurosurgery · hematology/oncology · ophthalmology · nephrology · pediatrics · pain management · psychiatry · other
`procedure_performed`: Was a surgical or invasive procedure done?	yes / no	gives P(yes)

What we found

1. Training made it much more accurate

Tested on 354 notes the model never saw. Each number is the share of notes answered correctly.

Table with columns: Question, Before training, After training, Mistakes removed
Question	Before training	After training	Mistakes removed
Document type	65.0%	94.9%	85% fewer mistakes
Specialty	74.6%	91.1%	65% fewer mistakes
Procedure performed?	83.9%	97.5%	84% fewer mistakes

Accuracy before vs after training

These gains are not luck. Even at the cautious end of the 95% confidence range, the improvement is at least +25, +12 and +10 points.

The biggest fixes:

Consult notes: recognised 7% → 93% of the time. Before training, the model called 142 notes "progress notes" when only 28 really were.
Discharge summaries: 46% → 100%.
Pain-management notes: 14% → 100%.
General-medicine notes: 37% → 76% (still the hardest specialty).

Document type accuracy per class, before vs after

Specialty accuracy per class, before vs after

2. Its confidence can be trusted, so most notes can be automated

When the model is very confident (95% or more), it is almost always right: every such test answer for document type and specialty was correct. At medium confidence it is, if anything, too modest. That lets you automate the confident cases and send only the uncertain ones to a person.

Table with columns: Question, Notes that can be decided automatically at ≤5% error: before → after
Question	Notes that can be decided automatically at ≤5% error: before → after
Document type	44% → 99%
Specialty	24% → 91%
Procedure performed?	48% → 100%

Share of notes decided automatically at ≤5% error

Stated confidence vs actual accuracy

3. Reading the answer beats writing it

We asked the same 50 test notes (148 questions) three ways, on the same GPU:

Table with columns: Approach, Correct, Words written, Time per question
Approach	Correct	Words written	Time per question
Normal chat model (Qwen3.5-2B) writes its answer, then code parses it	85.1%	4 tokens	453 ms
Untrained base model, read in one step	80.4%	0	298 ms
This model, read in one step	98.0%	0	257 ms

What this shows:

The speed comes from reading instead of writing: 1.8× faster.
The accuracy comes from our training: the untrained model, read the same way, is less accurate than the chatbot.

You need both, and this model has both. (The chart at the top of this card shows this comparison.)

4. Training was worth it compared with cheaper shortcuts

We also tried quick fixes that need no real training: correcting the untrained model's bias, or fitting a small classifier on its internal features with 300 labelled examples. The best shortcut reached 87% / 82% / 95%. This model reaches 95% / 91% / 97.5%, so it is 8, 9 and 2 points better, and it can automate far more notes (99% vs 73% for document type).

Training vs cheap shortcuts

Fair note: this model was trained on about 1,550 labelled examples per question, while the shortcuts used 300.

5. Speed and deployment tip

Measured on an NVIDIA T4 GPU, one question at a time, with notes averaging about 770 tokens:

Table with columns: Version, Time per question, Time per note (all 3 questions)
Version	Time per question	Time per note (all 3 questions)
Add-on loaded separately (default)	311 ms	945 ms
Add-on merged into the model	257 ms	794 ms

Time per question on a T4

Tip: merge the add-on before serving with model = model.merge_and_unload(). It gives the same answers (100% identical choices) and is faster.

The T4 test machine lacked the optimised attention kernels (flash-linear-attention, causal-conv1d), so a properly set-up GPU will be faster still.

Honest limitations

Sample data, not real hospitals. MTSamples are de-identified example transcriptions, filed into website categories. Real clinical notes look different, so expect lower accuracy there. Not for clinical use.
"Procedure performed" is an easy question. Its labels were partly derived from note headers such as "PROCEDURE PERFORMED:", so the near-perfect score mostly shows the model learned that rule.
Part of the document-type gain comes from fixing a bias. The untrained model over-used "progress note"; cheap fixes recover part of that gap (65% → 87%).
Small test set (331–354 notes per question): results can move a few points either way.
Specialty probabilities are slightly cautious. When the model says 50–70%, it is actually right about 77% of the time.
Long notes are cut at 1,024 tokens (about 20% of notes). On this test set, that did not reduce accuracy.
It only knows these three questions. It has not been tested on new questions.

Glossary

Table with columns: Term, Meaning
Term	Meaning
Jev / System One model	A model that picks from a fixed list of answers with probabilities in one step, instead of writing text
LoRA adapter	A small add-on trained on top of a big model; cheap to train and tiny to store
Calibrated	When it says 90%, it is right about 90% of the time
Auto-decidable at ≤5% error	The share of notes you can let the model decide alone, most confident first, while keeping mistakes at or below 5%
Macro-F1	Accuracy averaged fairly across all classes, so rare classes count as much as common ones
ECE	Calibration error, i.e. how far stated confidence is from real accuracy (lower is better)

Full numbers: results/ (eval_*.json, predictions_*.jsonl, benchmark_jev.md, diagnostics_summary.json). The charts are made by make_charts.py from those numbers.

Usage

Needs a GPU (a free Colab T4 works). This repo is private while it is in review: log in first with huggingface-cli login, or pass token= to snapshot_download.

python

import json, torchfrom huggingface_hub import snapshot_downloadfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer path = snapshot_download("theshekslaw/clinical-jev-qwen3.5-2b-mtsamples")      # add token="hf_..." if not logged incfg = json.load(open(f"{path}/training_config.json"))                        # base model, revision, prompt, max note lengthquestions = json.load(open(f"{path}/questions.json"))                        # the 3 typed questions and their optionstemps = json.load(open(f"{path}/calibration.json"))                          # one temperature per question (fitted on validation) dtype = torch.bfloat16 if torch.cuda.get_device_capability()[0] >= 8 else torch.float16   # T4 -> fp16 (as evaluated)tok = AutoTokenizer.from_pretrained(cfg["BASE_MODEL"], revision=cfg["BASE_REVISION"])model = AutoModelForCausalLM.from_pretrained(cfg["BASE_MODEL"], revision=cfg["BASE_REVISION"], dtype=dtype)model = PeftModel.from_pretrained(model, path).merge_and_unload().cuda().eval()   # merged: same answers, faster  @torch.no_grad()def ask(note, q):    """One forward pass, nothing generated: a probability for every allowed option (Jev-style answer)."""    opts = "\n".join(f"{chr(65 + i)}. {desc}" for i, (_, desc) in enumerate(q["options"]))    suffix = f"\n\n### Question\n{q['text']}\n\n### Options\n{opts}\n\n### Answer:"     # exactly the training prompt    note_ids = tok.encode(note, add_special_tokens=False)[: cfg["MAX_NOTE_TOKENS"]]    suffix_ids = tok.encode(suffix, add_special_tokens=False)    ids = tok.encode(cfg["INSTRUCTION"], add_special_tokens=False) + note_ids + suffix_ids    letters = []    for i in range(len(q["options"])):                                       # token id of " A", " B", ... after the prompt        full = tok.encode(suffix + " " + chr(65 + i), add_special_tokens=False)        assert full[:-1] == suffix_ids, "option letter is not a single token"        letters.append(full[-1])    logits = model(torch.tensor([ids]).cuda(), logits_to_keep=1).logits[0, -1, letters].float()    p = torch.softmax(logits / temps[q["id"]], -1).tolist()    probs = {key: round(v, 4) for (key, _), v in zip(q["options"], p)}    if q["type"] == "noul":                                                  # yes/no question -> P(yes)        return {"type": "noul", "noul": probs["yes"]}    best = max(probs, key=probs.get)    return {"type": "choice", "choice": best, "confidence": probs[best], "probabilities": probs}  note = ("PREOPERATIVE DIAGNOSIS: Cataract, right eye. POSTOPERATIVE DIAGNOSIS: Same. "        "PROCEDURE: Phacoemulsification with intraocular lens implantation, right eye. "        "ANESTHESIA: Topical with MAC. The patient tolerated the procedure well.")answers = {q["id"]: ask(note, q) for q in questions}print(json.dumps(answers, indent=1))

Example output for that note (from the evaluation run; probabilities trimmed):

json

{ "doc_type":  {"type": "choice", "choice": "operative_procedure_note", "confidence": 0.9912, "probabilities": {"operative_procedure_note": 0.9912, "consult_history_physical": 0.0043, "...": "..."}}, "specialty": {"type": "choice", "choice": "ophthalmology", "confidence": 0.9904, "probabilities": {"ophthalmology": 0.9904, "...": "..."}}, "procedure_performed": {"type": "noul", "noul": 0.9935}}

How to use the probabilities:

Act automatically when the confidence (or P(yes) / P(no)) is high.
Send the rest to a person. The "decided automatically at ≤5% error" numbers above come from exactly this rule.

Training details

Add-on: LoRA r=16, alpha=32, dropout 0.05, on q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, in_proj_qkv, in_proj_z, in_proj_a, in_proj_b and out_proj.
Schedule: 2 epochs; lr 0.0001 (cosine, 5% warm-up); effective batch 16; notes truncated to 1,024 tokens; precision fp16 on a T4.
Option order was shuffled during training, so the model learns the meaning of each option rather than its position.
Loss: cross-entropy between the option-letter probabilities and the (soft) gold answer.
Selection and calibration: the checkpoint with the best validation loss was kept, and one temperature per question was fitted on validation.
Data: theshekslaw/mtsamples-typed-decisions: 1,649 train / 354 validation / 354 test notes, de-duplicated so no note appears in two splits.

In one minute

It does this in one step, without writing any text. That makes it:

Accurate: 95% on document type, 91% on specialty and 97.5% on "was a procedure performed?". The same model scored 65%, 75% and 84% before training.
Fast: 1.8× faster than asking a normal chatbot-style model to write its answer.
Safe to automate: the probabilities can be trusted. 99% of notes can be sorted by document type automatically while staying under 5% error, against 44% before training. The rest go to a person.
Always well-formed: it can only pick one of the options you give it, so the answer never needs parsing and can never be malformed.

Reading the answer is faster; training makes it right

The idea: "Jev-style" decisions instead of chatting

Most AI systems answer a question by writing text ("The answer is B…"), which your code then has to parse. That is slow, and the answer can come back in the wrong format.

This model follows the Jev / "System One" idea:

text

                     normal chatbot                      this model (Jev-style)note + question ──►  writes "B" token by token  ──►      reads the model's preference for A, B, C… in ONE step                     then code parses the text            ──► {A: 2%, B: 95%, C: 3%}  (nothing to parse)

                     normal chatbot                      this model (Jev-style)note + question ──►  writes "B" token by token  ──►      reads the model's preference for A, B, C… in ONE step                     then code parses the text            ──► {A: 2%, B: 95%, C: 3%}  (nothing to parse)

The three questions it answers

Table with columns: Question, Kind, Possible answers
Question	Kind	Possible answers
`doc_type`: What type of document is this?	pick one of 7	operative/procedure note · consult or history & physical · radiology/diagnostic report · progress/SOAP note · discharge summary · ER report · other (office note, letter, etc.)
`specialty`: Which specialty does it belong to?	pick one of 16	cardiology/pulmonary · orthopedics · general medicine · GI · neurology · urology · OB/GYN · ENT · neurosurgery · hematology/oncology · ophthalmology · nephrology · pediatrics · pain management · psychiatry · other
`procedure_performed`: Was a surgical or invasive procedure done?	yes / no	gives P(yes)

What we found

1. Training made it much more accurate

Tested on 354 notes the model never saw. Each number is the share of notes answered correctly.

Table with columns: Question, Before training, After training, Mistakes removed
Question	Before training	After training	Mistakes removed
Document type	65.0%	94.9%	85% fewer mistakes
Specialty	74.6%	91.1%	65% fewer mistakes
Procedure performed?	83.9%	97.5%	84% fewer mistakes

Accuracy before vs after training

These gains are not luck. Even at the cautious end of the 95% confidence range, the improvement is at least +25, +12 and +10 points.

The biggest fixes:

Consult notes: recognised 7% → 93% of the time. Before training, the model called 142 notes "progress notes" when only 28 really were.
Discharge summaries: 46% → 100%.
Pain-management notes: 14% → 100%.
General-medicine notes: 37% → 76% (still the hardest specialty).

Document type accuracy per class, before vs after

Specialty accuracy per class, before vs after

2. Its confidence can be trusted, so most notes can be automated

Table with columns: Question, Notes that can be decided automatically at ≤5% error: before → after
Question	Notes that can be decided automatically at ≤5% error: before → after
Document type	44% → 99%
Specialty	24% → 91%
Procedure performed?	48% → 100%

Share of notes decided automatically at ≤5% error

Stated confidence vs actual accuracy

3. Reading the answer beats writing it

We asked the same 50 test notes (148 questions) three ways, on the same GPU:

Table with columns: Approach, Correct, Words written, Time per question
Approach	Correct	Words written	Time per question
Normal chat model (Qwen3.5-2B) writes its answer, then code parses it	85.1%	4 tokens	453 ms
Untrained base model, read in one step	80.4%	0	298 ms
This model, read in one step	98.0%	0	257 ms

What this shows:

The speed comes from reading instead of writing: 1.8× faster.
The accuracy comes from our training: the untrained model, read the same way, is less accurate than the chatbot.

You need both, and this model has both. (The chart at the top of this card shows this comparison.)

4. Training was worth it compared with cheaper shortcuts

Training vs cheap shortcuts

Fair note: this model was trained on about 1,550 labelled examples per question, while the shortcuts used 300.

5. Speed and deployment tip

Measured on an NVIDIA T4 GPU, one question at a time, with notes averaging about 770 tokens:

Table with columns: Version, Time per question, Time per note (all 3 questions)
Version	Time per question	Time per note (all 3 questions)
Add-on loaded separately (default)	311 ms	945 ms
Add-on merged into the model	257 ms	794 ms

Time per question on a T4

Tip: merge the add-on before serving with model = model.merge_and_unload(). It gives the same answers (100% identical choices) and is faster.

The T4 test machine lacked the optimised attention kernels (flash-linear-attention, causal-conv1d), so a properly set-up GPU will be faster still.

Honest limitations

Sample data, not real hospitals. MTSamples are de-identified example transcriptions, filed into website categories. Real clinical notes look different, so expect lower accuracy there. Not for clinical use.
"Procedure performed" is an easy question. Its labels were partly derived from note headers such as "PROCEDURE PERFORMED:", so the near-perfect score mostly shows the model learned that rule.
Part of the document-type gain comes from fixing a bias. The untrained model over-used "progress note"; cheap fixes recover part of that gap (65% → 87%).
Small test set (331–354 notes per question): results can move a few points either way.
Specialty probabilities are slightly cautious. When the model says 50–70%, it is actually right about 77% of the time.
Long notes are cut at 1,024 tokens (about 20% of notes). On this test set, that did not reduce accuracy.
It only knows these three questions. It has not been tested on new questions.

Glossary

Table with columns: Term, Meaning
Term	Meaning
Jev / System One model	A model that picks from a fixed list of answers with probabilities in one step, instead of writing text
LoRA adapter	A small add-on trained on top of a big model; cheap to train and tiny to store
Calibrated	When it says 90%, it is right about 90% of the time
Auto-decidable at ≤5% error	The share of notes you can let the model decide alone, most confident first, while keeping mistakes at or below 5%
Macro-F1	Accuracy averaged fairly across all classes, so rare classes count as much as common ones
ECE	Calibration error, i.e. how far stated confidence is from real accuracy (lower is better)

Full numbers: results/ (eval_*.json, predictions_*.jsonl, benchmark_jev.md, diagnostics_summary.json). The charts are made by make_charts.py from those numbers.

Usage

Needs a GPU (a free Colab T4 works). This repo is private while it is in review: log in first with huggingface-cli login, or pass token= to snapshot_download.

python

import json, torchfrom huggingface_hub import snapshot_downloadfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer path = snapshot_download("theshekslaw/clinical-jev-qwen3.5-2b-mtsamples")      # add token="hf_..." if not logged incfg = json.load(open(f"{path}/training_config.json"))                        # base model, revision, prompt, max note lengthquestions = json.load(open(f"{path}/questions.json"))                        # the 3 typed questions and their optionstemps = json.load(open(f"{path}/calibration.json"))                          # one temperature per question (fitted on validation) dtype = torch.bfloat16 if torch.cuda.get_device_capability()[0] >= 8 else torch.float16   # T4 -> fp16 (as evaluated)tok = AutoTokenizer.from_pretrained(cfg["BASE_MODEL"], revision=cfg["BASE_REVISION"])model = AutoModelForCausalLM.from_pretrained(cfg["BASE_MODEL"], revision=cfg["BASE_REVISION"], dtype=dtype)model = PeftModel.from_pretrained(model, path).merge_and_unload().cuda().eval()   # merged: same answers, faster  @torch.no_grad()def ask(note, q):    """One forward pass, nothing generated: a probability for every allowed option (Jev-style answer)."""    opts = "\n".join(f"{chr(65 + i)}. {desc}" for i, (_, desc) in enumerate(q["options"]))    suffix = f"\n\n### Question\n{q['text']}\n\n### Options\n{opts}\n\n### Answer:"     # exactly the training prompt    note_ids = tok.encode(note, add_special_tokens=False)[: cfg["MAX_NOTE_TOKENS"]]    suffix_ids = tok.encode(suffix, add_special_tokens=False)    ids = tok.encode(cfg["INSTRUCTION"], add_special_tokens=False) + note_ids + suffix_ids    letters = []    for i in range(len(q["options"])):                                       # token id of " A", " B", ... after the prompt        full = tok.encode(suffix + " " + chr(65 + i), add_special_tokens=False)        assert full[:-1] == suffix_ids, "option letter is not a single token"        letters.append(full[-1])    logits = model(torch.tensor([ids]).cuda(), logits_to_keep=1).logits[0, -1, letters].float()    p = torch.softmax(logits / temps[q["id"]], -1).tolist()    probs = {key: round(v, 4) for (key, _), v in zip(q["options"], p)}    if q["type"] == "noul":                                                  # yes/no question -> P(yes)        return {"type": "noul", "noul": probs["yes"]}    best = max(probs, key=probs.get)    return {"type": "choice", "choice": best, "confidence": probs[best], "probabilities": probs}  note = ("PREOPERATIVE DIAGNOSIS: Cataract, right eye. POSTOPERATIVE DIAGNOSIS: Same. "        "PROCEDURE: Phacoemulsification with intraocular lens implantation, right eye. "        "ANESTHESIA: Topical with MAC. The patient tolerated the procedure well.")answers = {q["id"]: ask(note, q) for q in questions}print(json.dumps(answers, indent=1))

import json, torchfrom huggingface_hub import snapshot_downloadfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer path = snapshot_download("theshekslaw/clinical-jev-qwen3.5-2b-mtsamples")      # add token="hf_..." if not logged incfg = json.load(open(f"{path}/training_config.json"))                        # base model, revision, prompt, max note lengthquestions = json.load(open(f"{path}/questions.json"))                        # the 3 typed questions and their optionstemps = json.load(open(f"{path}/calibration.json"))                          # one temperature per question (fitted on validation) dtype = torch.bfloat16 if torch.cuda.get_device_capability()[0] >= 8 else torch.float16   # T4 -> fp16 (as evaluated)tok = AutoTokenizer.from_pretrained(cfg["BASE_MODEL"], revision=cfg["BASE_REVISION"])model = AutoModelForCausalLM.from_pretrained(cfg["BASE_MODEL"], revision=cfg["BASE_REVISION"], dtype=dtype)model = PeftModel.from_pretrained(model, path).merge_and_unload().cuda().eval()   # merged: same answers, faster  @torch.no_grad()def ask(note, q):    """One forward pass, nothing generated: a probability for every allowed option (Jev-style answer)."""    opts = "\n".join(f"{chr(65 + i)}. {desc}" for i, (_, desc) in enumerate(q["options"]))    suffix = f"\n\n### Question\n{q['text']}\n\n### Options\n{opts}\n\n### Answer:"     # exactly the training prompt    note_ids = tok.encode(note, add_special_tokens=False)[: cfg["MAX_NOTE_TOKENS"]]    suffix_ids = tok.encode(suffix, add_special_tokens=False)    ids = tok.encode(cfg["INSTRUCTION"], add_special_tokens=False) + note_ids + suffix_ids    letters = []    for i in range(len(q["options"])):                                       # token id of " A", " B", ... after the prompt        full = tok.encode(suffix + " " + chr(65 + i), add_special_tokens=False)        assert full[:-1] == suffix_ids, "option letter is not a single token"        letters.append(full[-1])    logits = model(torch.tensor([ids]).cuda(), logits_to_keep=1).logits[0, -1, letters].float()    p = torch.softmax(logits / temps[q["id"]], -1).tolist()    probs = {key: round(v, 4) for (key, _), v in zip(q["options"], p)}    if q["type"] == "noul":                                                  # yes/no question -> P(yes)        return {"type": "noul", "noul": probs["yes"]}    best = max(probs, key=probs.get)    return {"type": "choice", "choice": best, "confidence": probs[best], "probabilities": probs}  note = ("PREOPERATIVE DIAGNOSIS: Cataract, right eye. POSTOPERATIVE DIAGNOSIS: Same. "        "PROCEDURE: Phacoemulsification with intraocular lens implantation, right eye. "        "ANESTHESIA: Topical with MAC. The patient tolerated the procedure well.")answers = {q["id"]: ask(note, q) for q in questions}print(json.dumps(answers, indent=1))

Example output for that note (from the evaluation run; probabilities trimmed):

json

{ "doc_type":  {"type": "choice", "choice": "operative_procedure_note", "confidence": 0.9912, "probabilities": {"operative_procedure_note": 0.9912, "consult_history_physical": 0.0043, "...": "..."}}, "specialty": {"type": "choice", "choice": "ophthalmology", "confidence": 0.9904, "probabilities": {"ophthalmology": 0.9904, "...": "..."}}, "procedure_performed": {"type": "noul", "noul": 0.9935}}

{ "doc_type":  {"type": "choice", "choice": "operative_procedure_note", "confidence": 0.9912, "probabilities": {"operative_procedure_note": 0.9912, "consult_history_physical": 0.0043, "...": "..."}}, "specialty": {"type": "choice", "choice": "ophthalmology", "confidence": 0.9904, "probabilities": {"ophthalmology": 0.9904, "...": "..."}}, "procedure_performed": {"type": "noul", "noul": 0.9935}}

How to use the probabilities:

Act automatically when the confidence (or P(yes) / P(no)) is high.
Send the rest to a person. The "decided automatically at ≤5% error" numbers above come from exactly this rule.

Training details

Add-on: LoRA r=16, alpha=32, dropout 0.05, on q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, in_proj_qkv, in_proj_z, in_proj_a, in_proj_b and out_proj.
Schedule: 2 epochs; lr 0.0001 (cosine, 5% warm-up); effective batch 16; notes truncated to 1,024 tokens; precision fp16 on a T4.
Option order was shuffled during training, so the model learns the meaning of each option rather than its position.
Loss: cross-entropy between the option-letter probabilities and the (soft) gold answer.
Selection and calibration: the checkpoint with the best validation loss was kept, and one temperature per question was fitted on validation.
Data: theshekslaw/mtsamples-typed-decisions: 1,649 train / 354 validation / 354 test notes, de-duplicated so no note appears in two splits.

clinical-jev-qwen3.5-2b-mtsamples

README

In one minute

The idea: "Jev-style" decisions instead of chatting

The three questions it answers

What we found

1. Training made it much more accurate

2. Its confidence can be trusted, so most notes can be automated

3. Reading the answer beats writing it

4. Training was worth it compared with cheaper shortcuts

5. Speed and deployment tip

Honest limitations

Glossary

Usage

Training details

Explore FriendliAI today

README

In one minute

The idea: "Jev-style" decisions instead of chatting

The three questions it answers

What we found

1. Training made it much more accurate

2. Its confidence can be trusted, so most notes can be automated

3. Reading the answer beats writing it

4. Training was worth it compared with cheaper shortcuts

5. Speed and deployment tip

Honest limitations

Glossary

Usage

Training details