hiimivantang

qwen3-8b-jmed-lora

README

License: apache-2.0

Base model (required)

Table

Base model	`Qwen/Qwen3-8B`
Revision (commit)	`b968826d9c46dd6066d109eabc6255188de91218`
Chat template	`qwen3`, thinking disabled (`enable_thinking=False`)

The adapter was trained against this exact revision. adapter_config.json records it, and AutoPeftModelForCausalLM uses it; if you load the base model yourself, pass revision="b968826d9c46dd6066d109eabc6255188de91218" as below.

Usage

Transformers + PEFT

python

import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer BASE, REV = "Qwen/Qwen3-8B", "b968826d9c46dd6066d109eabc6255188de91218"tok = AutoTokenizer.from_pretrained(BASE, revision=REV)model = AutoModelForCausalLM.from_pretrained(BASE, revision=REV, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "hiimivantang/qwen3-8b-jmed-lora") messages = [{"role": "system", "content": SYSTEM_PROMPT},            {"role": "user", "content": USER_PROMPT}]  # one of the task templates belowinputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,                                 return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=1024, do_sample=False, repetition_penalty=1.1)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

vLLM

bash

vllm serve Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218 --enable-lora --max-lora-rank 16 \  --lora-modules jmed-lora=hiimivantang/qwen3-8b-jmed-lora

Request "model": "jmed-lora" with temperature 0 and repetition_penalty 1.1 (see Limitations).

Prompt format

The adapter was trained on exactly these prompts (Japanese). Use them verbatim; other prompts will not reproduce the results below. Replace {note} with the document text.

System prompt (all tasks)

text

あなたは日本語の医療文書を扱う情報抽出アシスタントです。指示に従い、文書に書かれている内容だけを根拠に回答してください。診断や治療の推奨は行わないでください。

Task A — entity extraction

text

次の医療文書から、患者本人に現在認められる「症状」「疾患」「薬剤」を抽出してください。規則:- 文書中の表記をそのまま抜き出す（言い換え・正規化をしない）。- symptoms: 患者本人に認められる症状・所見（「〜なし」「〜は認めない」など否定されたものは含めない）。- diseases: 患者本人が罹患している（既往を含む）診断済みの疾患。疑い病名・家族歴の疾患は含めない。- medications: 患者が使用中または使用した薬剤名（用量は含めない）。- 該当がなければ空リストにする。出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"symptoms": [...], "diseases": [...], "medications": [...]} 文書:「{note}」

Task B — document type

text

次の医療文書の種類を、以下の選択肢から1つ選んでください。選択肢: 外来診療録, 入院時記録, 退院時サマリー, 紹介状, 看護記録, 検査・画像所見報告書出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"label": "<選択肢のいずれか>"} 文書:「{note}」

Task C — structured record

text

次の医療文書から、以下の項目を構造化して抽出してください。規則:- age: 年齢（整数）。記載がなければ null。- sex: "男性" または "女性"。記載がなければ null。- chief_complaint: 主訴（文書中の表記）。記載がなければ null。- symptoms: 患者本人に認められる症状（否定されたものは含めない）。- negated_symptoms: 「〜なし」「〜は認めない」など否定されている症状。- past_history: 既往歴の疾患。- family_history: 家族歴の疾患（続柄は含めない）。- medications: 使用中の薬剤名（用量は含めない）。- allergies: アレルギーの原因物質・薬剤。- diagnoses: 現在の確定診断名。- suspected_diagnosis: 疑い・可能性として記載されている病名。- リスト項目は文書中の表記をそのまま抜き出し、該当がなければ空リストにする。出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"age": ..., "sex": ..., "chief_complaint": ..., "symptoms": ..., "negated_symptoms": ..., "past_history": ..., "family_history": ..., "medications": ..., "allergies": ..., "diagnoses": ..., "suspected_diagnosis": ...} 文書:「{note}」

The model replies with a single JSON object.

Evaluation

Held-out synthetic test set (223 notes per task, 25% of vocabulary unseen in training), greedy decoding, thinking disabled, scored by deterministic code (no LLM judge):

Table with columns: Task, Qwen3-8B (base, same prompt), This adapter, GPT-4.1, GPT-6.1-sol*
Task	Qwen3-8B (base, same prompt)	This adapter	GPT-4.1	GPT-6.1-sol*
A — entity extraction (strict F1)	0.951	0.997	0.962	0.949
B — document type (macro-F1)	0.622	0.988	0.711	0.761
C — structured record (field accuracy)	0.906	0.993	0.902	0.974
C — records with all 11 fields correct	35%	94%	36%	74%

* gpt-6.1-sol was added on 2026-10-30 as a post-hoc comparison (after the original results were known). The adapter beats it on every task above, by wider margins than vs GPT-4.1; it also beats it on the 215 notes written by a different generator (e.g. document type 0.789 vs 0.594).

Different writing style: the gains held on 215 notes written by a different generator (NVIDIA Nemotron), though smaller for document type (macro-F1 0.988 → 0.789; base 0.335, GPT-4.1 0.556).
Real documents (JMED-LLM, human-annotated case reports, 100 items): disease extraction strict F1 0.265 → 0.473 (GPT-4.1: 0.110; gpt-6.1-sol: 0.392); under lenient span matching GPT-4.1 (0.71) and gpt-6.1-sol (0.78) are ahead of the adapter (0.65).
It is a specialist: on real medical tasks it was not trained for (adverse-drug-event certainty, cancer staging) it did no better than the base model, and OpenAI models were clearly better (staging 0.91–0.97 vs 0.69). General Japanese ability was not degraded: JCommonsenseQA −0.6, JNLI +1.2 and JMMLU-Med +6.0 points versus the base model, all within the pre-registered −2-point non-inferiority margin.

Training

Table

Method	LoRA (PEFT), LLaMA-Factory
Rank / alpha / dropout	16 / 32 / 0.05
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`
Learning rate / schedule	1e-4, cosine, 3% warm-up
Batch	4 × gradient accumulation 4
Epochs / steps	2 / 520
Released checkpoint	step 300 (lowest validation loss, 0.00344; chosen by a pre-registered rule)
Seed / precision	42 / bf16
Hardware	NVIDIA DGX Spark (GB10), 2.6 h

Training metrics are in all_results.json, train_results.json, eval_results.json and trainer_state.json.

Training data

Synthetic clinical notes of fictional patients (no real patient information), generated locally by Qwen/Qwen3.8-27B-FP8 (Apache-2.0) from sampled specifications; a deterministic validator rejected any note whose text did not support its labels. 1,367 training + 135 validation notes (4,151 / 405 task records).
47 real documents from the NTCIR-16 Real-MedNLP test collection (MedTxt-CR-JA), CC BY 4.0, after removing every document overlapping the JMED-LLM evaluation set.
No OpenAI output was used for training. Evaluation-only data (JMED-LLM, JGLUE, JMMLU) was not trained on.

Attribution

This adapter was trained in part on documents from the NTCIR-16 Real-MedNLP test collection, licensed under CC BY 4.0:

S. Yada, Y. Nakamura, S. Wakamiya, E. Aramaki. Real-MedNLP: Overview of REAL document-based MEDical Natural Language Processing Task. NTCIR-16, 2022. https://sociocom.naist.jp/download/real-mednlp-test-collection/

The documents were converted into training examples; they are not redistributed here.

Conclusions (both experiments)

This adapter is one half of a two-experiment study of small task-specific LoRAs on Qwen3-8B. Across both:

Task-tuning works and is economical. On the tasks they were trained for, the adapters matched or beat GPT-4.1 and gpt-6.1-sol (see the tables above) — a 61 MB adapter on an 8B base.
They are specialists. On medical tasks outside their training (cancer staging, adverse-event grading) they did no better than the base model, and frontier models were clearly better. Do not expect transfer to neighbouring tasks.
They do not improve retrieval-augmented pipelines. In the companion experiment (question-understanding, grounded-answering), retrieval quality dominated the end-to-end result, and the adapters added nothing (the answering adapter made the full pipeline slightly worse).
Both adapters serve together on one shared base (vLLM --enable-lora or any PEFT stack); they were trained on the same pinned base revision and are never merged.

Cross-language note (base model)

As a post-hoc check (2026-10-30), the base Qwen3-8B was run on an English translation of the same test sets (each note translated with its gold labels, every label verified to appear verbatim in the English text; the adapter was not trained or tested on English). The base model is slightly better on these tasks in Japanese than on the identical content in English:

Table with columns: Base Qwen3-8B, Japanese, English
Base Qwen3-8B	Japanese	English
Entity extraction, strict F1 (synthetic test)	0.951	0.938
Document type, macro-F1 (synthetic test)	0.622	0.563
Structured record, field accuracy (synthetic test)	0.906	0.879
Disease extraction, strict F1 (real case reports)	0.265	0.150

The differences are small (1–5 points, significant on structured extraction and the cross-generator set); the large drop on real case reports reflects that they are Japanese clinical documents, so exact clinical phrasing is lost in translation. This says nothing directly about the adapter, which was trained on Japanese only.

Limitations

Synthetic in-distribution gains overstate real-world gains. The main test set shares the training generator; part of the gain is learning this project's labelling conventions (what counts as a symptom, how a span is written). Measure it on your own documents.
Little real training text: only 47 real documents survived the leakage filter.
Repetition loops on long real documents: with plain greedy decoding the adapter looped until the 1,024-token limit on 37 of 400 real external documents (the base model: 1). A repetition penalty of 1.1 cut this to 7 of 400, which is why the usage examples set it. The penalty was tested after the fact, not pre-registered.
Specialist only: no gain on medical tasks outside the three it was trained on.
Single training seed; not clinically validated.

License

The adapter weights are released under Apache-2.0 (a derivative of Apache-2.0 Qwen/Qwen3-8B). Training data attribution as above. The base model has its own licence; see Qwen/Qwen3-8B.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

hiimivantang

Model Tree

Base

Qwen/Qwen3-8B

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Base model (required)

Table

Base model	`Qwen/Qwen3-8B`
Revision (commit)	`b968826d9c46dd6066d109eabc6255188de91218`
Chat template	`qwen3`, thinking disabled (`enable_thinking=False`)

Usage

Transformers + PEFT

python

import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer BASE, REV = "Qwen/Qwen3-8B", "b968826d9c46dd6066d109eabc6255188de91218"tok = AutoTokenizer.from_pretrained(BASE, revision=REV)model = AutoModelForCausalLM.from_pretrained(BASE, revision=REV, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "hiimivantang/qwen3-8b-jmed-lora") messages = [{"role": "system", "content": SYSTEM_PROMPT},            {"role": "user", "content": USER_PROMPT}]  # one of the task templates belowinputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,                                 return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=1024, do_sample=False, repetition_penalty=1.1)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer BASE, REV = "Qwen/Qwen3-8B", "b968826d9c46dd6066d109eabc6255188de91218"tok = AutoTokenizer.from_pretrained(BASE, revision=REV)model = AutoModelForCausalLM.from_pretrained(BASE, revision=REV, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "hiimivantang/qwen3-8b-jmed-lora") messages = [{"role": "system", "content": SYSTEM_PROMPT},            {"role": "user", "content": USER_PROMPT}]  # one of the task templates belowinputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,                                 return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=1024, do_sample=False, repetition_penalty=1.1)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

vLLM

bash

vllm serve Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218 --enable-lora --max-lora-rank 16 \  --lora-modules jmed-lora=hiimivantang/qwen3-8b-jmed-lora

vllm serve Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218 --enable-lora --max-lora-rank 16 \  --lora-modules jmed-lora=hiimivantang/qwen3-8b-jmed-lora

Request "model": "jmed-lora" with temperature 0 and repetition_penalty 1.1 (see Limitations).

Prompt format

The adapter was trained on exactly these prompts (Japanese). Use them verbatim; other prompts will not reproduce the results below. Replace {note} with the document text.

System prompt (all tasks)

text

あなたは日本語の医療文書を扱う情報抽出アシスタントです。指示に従い、文書に書かれている内容だけを根拠に回答してください。診断や治療の推奨は行わないでください。

Task A — entity extraction

text

次の医療文書から、患者本人に現在認められる「症状」「疾患」「薬剤」を抽出してください。規則:- 文書中の表記をそのまま抜き出す（言い換え・正規化をしない）。- symptoms: 患者本人に認められる症状・所見（「〜なし」「〜は認めない」など否定されたものは含めない）。- diseases: 患者本人が罹患している（既往を含む）診断済みの疾患。疑い病名・家族歴の疾患は含めない。- medications: 患者が使用中または使用した薬剤名（用量は含めない）。- 該当がなければ空リストにする。出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"symptoms": [...], "diseases": [...], "medications": [...]}文書:「{note}」

次の医療文書から、患者本人に現在認められる「症状」「疾患」「薬剤」を抽出してください。規則:- 文書中の表記をそのまま抜き出す（言い換え・正規化をしない）。- symptoms: 患者本人に認められる症状・所見（「〜なし」「〜は認めない」など否定されたものは含めない）。- diseases: 患者本人が罹患している（既往を含む）診断済みの疾患。疑い病名・家族歴の疾患は含めない。- medications: 患者が使用中または使用した薬剤名（用量は含めない）。- 該当がなければ空リストにする。出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"symptoms": [...], "diseases": [...], "medications": [...]} 文書:「{note}」

Task B — document type

text

次の医療文書の種類を、以下の選択肢から1つ選んでください。選択肢: 外来診療録, 入院時記録, 退院時サマリー, 紹介状, 看護記録, 検査・画像所見報告書出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"label": "<選択肢のいずれか>"}文書:「{note}」

次の医療文書の種類を、以下の選択肢から1つ選んでください。選択肢: 外来診療録, 入院時記録, 退院時サマリー, 紹介状, 看護記録, 検査・画像所見報告書出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"label": "<選択肢のいずれか>"} 文書:「{note}」

Task C — structured record

text

次の医療文書から、以下の項目を構造化して抽出してください。規則:- age: 年齢（整数）。記載がなければ null。- sex: "男性" または "女性"。記載がなければ null。- chief_complaint: 主訴（文書中の表記）。記載がなければ null。- symptoms: 患者本人に認められる症状（否定されたものは含めない）。- negated_symptoms: 「〜なし」「〜は認めない」など否定されている症状。- past_history: 既往歴の疾患。- family_history: 家族歴の疾患（続柄は含めない）。- medications: 使用中の薬剤名（用量は含めない）。- allergies: アレルギーの原因物質・薬剤。- diagnoses: 現在の確定診断名。- suspected_diagnosis: 疑い・可能性として記載されている病名。- リスト項目は文書中の表記をそのまま抜き出し、該当がなければ空リストにする。出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"age": ..., "sex": ..., "chief_complaint": ..., "symptoms": ..., "negated_symptoms": ..., "past_history": ..., "family_history": ..., "medications": ..., "allergies": ..., "diagnoses": ..., "suspected_diagnosis": ...}文書:「{note}」

次の医療文書から、以下の項目を構造化して抽出してください。規則:- age: 年齢（整数）。記載がなければ null。- sex: "男性" または "女性"。記載がなければ null。- chief_complaint: 主訴（文書中の表記）。記載がなければ null。- symptoms: 患者本人に認められる症状（否定されたものは含めない）。- negated_symptoms: 「〜なし」「〜は認めない」など否定されている症状。- past_history: 既往歴の疾患。- family_history: 家族歴の疾患（続柄は含めない）。- medications: 使用中の薬剤名（用量は含めない）。- allergies: アレルギーの原因物質・薬剤。- diagnoses: 現在の確定診断名。- suspected_diagnosis: 疑い・可能性として記載されている病名。- リスト項目は文書中の表記をそのまま抜き出し、該当がなければ空リストにする。出力はJSONオブジェクトのみとし、説明文やコードブロック記号は付けないでください。形式: {"age": ..., "sex": ..., "chief_complaint": ..., "symptoms": ..., "negated_symptoms": ..., "past_history": ..., "family_history": ..., "medications": ..., "allergies": ..., "diagnoses": ..., "suspected_diagnosis": ...} 文書:「{note}」

The model replies with a single JSON object.

Evaluation

Held-out synthetic test set (223 notes per task, 25% of vocabulary unseen in training), greedy decoding, thinking disabled, scored by deterministic code (no LLM judge):

Table with columns: Task, Qwen3-8B (base, same prompt), This adapter, GPT-4.1, GPT-6.1-sol*
Task	Qwen3-8B (base, same prompt)	This adapter	GPT-4.1	GPT-6.1-sol*
A — entity extraction (strict F1)	0.951	0.997	0.962	0.949
B — document type (macro-F1)	0.622	0.988	0.711	0.761
C — structured record (field accuracy)	0.906	0.993	0.902	0.974
C — records with all 11 fields correct	35%	94%	36%	74%

Different writing style: the gains held on 215 notes written by a different generator (NVIDIA Nemotron), though smaller for document type (macro-F1 0.988 → 0.789; base 0.335, GPT-4.1 0.556).
Real documents (JMED-LLM, human-annotated case reports, 100 items): disease extraction strict F1 0.265 → 0.473 (GPT-4.1: 0.110; gpt-6.1-sol: 0.392); under lenient span matching GPT-4.1 (0.71) and gpt-6.1-sol (0.78) are ahead of the adapter (0.65).
It is a specialist: on real medical tasks it was not trained for (adverse-drug-event certainty, cancer staging) it did no better than the base model, and OpenAI models were clearly better (staging 0.91–0.97 vs 0.69). General Japanese ability was not degraded: JCommonsenseQA −0.6, JNLI +1.2 and JMMLU-Med +6.0 points versus the base model, all within the pre-registered −2-point non-inferiority margin.

Training

Table

Method	LoRA (PEFT), LLaMA-Factory
Rank / alpha / dropout	16 / 32 / 0.05
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`
Learning rate / schedule	1e-4, cosine, 3% warm-up
Batch	4 × gradient accumulation 4
Epochs / steps	2 / 520
Released checkpoint	step 300 (lowest validation loss, 0.00344; chosen by a pre-registered rule)
Seed / precision	42 / bf16
Hardware	NVIDIA DGX Spark (GB10), 2.6 h

Training metrics are in all_results.json, train_results.json, eval_results.json and trainer_state.json.

Training data

Synthetic clinical notes of fictional patients (no real patient information), generated locally by Qwen/Qwen3.8-27B-FP8 (Apache-2.0) from sampled specifications; a deterministic validator rejected any note whose text did not support its labels. 1,367 training + 135 validation notes (4,151 / 405 task records).
47 real documents from the NTCIR-16 Real-MedNLP test collection (MedTxt-CR-JA), CC BY 4.0, after removing every document overlapping the JMED-LLM evaluation set.
No OpenAI output was used for training. Evaluation-only data (JMED-LLM, JGLUE, JMMLU) was not trained on.

Attribution

This adapter was trained in part on documents from the NTCIR-16 Real-MedNLP test collection, licensed under CC BY 4.0:

S. Yada, Y. Nakamura, S. Wakamiya, E. Aramaki. Real-MedNLP: Overview of REAL document-based MEDical Natural Language Processing Task. NTCIR-16, 2022. https://sociocom.naist.jp/download/real-mednlp-test-collection/

The documents were converted into training examples; they are not redistributed here.

Conclusions (both experiments)

This adapter is one half of a two-experiment study of small task-specific LoRAs on Qwen3-8B. Across both:

Task-tuning works and is economical. On the tasks they were trained for, the adapters matched or beat GPT-4.1 and gpt-6.1-sol (see the tables above) — a 61 MB adapter on an 8B base.
They are specialists. On medical tasks outside their training (cancer staging, adverse-event grading) they did no better than the base model, and frontier models were clearly better. Do not expect transfer to neighbouring tasks.
They do not improve retrieval-augmented pipelines. In the companion experiment (question-understanding, grounded-answering), retrieval quality dominated the end-to-end result, and the adapters added nothing (the answering adapter made the full pipeline slightly worse).
Both adapters serve together on one shared base (vLLM --enable-lora or any PEFT stack); they were trained on the same pinned base revision and are never merged.

Cross-language note (base model)

Table with columns: Base Qwen3-8B, Japanese, English
Base Qwen3-8B	Japanese	English
Entity extraction, strict F1 (synthetic test)	0.951	0.938
Document type, macro-F1 (synthetic test)	0.622	0.563
Structured record, field accuracy (synthetic test)	0.906	0.879
Disease extraction, strict F1 (real case reports)	0.265	0.150

Limitations

Synthetic in-distribution gains overstate real-world gains. The main test set shares the training generator; part of the gain is learning this project's labelling conventions (what counts as a symptom, how a span is written). Measure it on your own documents.
Little real training text: only 47 real documents survived the leakage filter.
Repetition loops on long real documents: with plain greedy decoding the adapter looped until the 1,024-token limit on 37 of 400 real external documents (the base model: 1). A repetition penalty of 1.1 cut this to 7 of 400, which is why the usage examples set it. The penalty was tested after the fact, not pre-registered.
Specialist only: no gain on medical tasks outside the three it was trained on.
Single training seed; not clinically validated.

License

The adapter weights are released under Apache-2.0 (a derivative of Apache-2.0 Qwen/Qwen3-8B). Training data attribution as above. The base model has its own licence; see Qwen/Qwen3-8B.