Training
- Base:
Qwen/Qwen2.5-1.5B-Instruct (fp16)
- Data: synthetic flow/culture instances whose raw-input targets are reused
from the reading gold set (
labwright/extract/synthetic.py) — n_train 3268,
n_eval 364
- LoRA: r/LoRA from the adapter config (
lora_alpha 32, PEFT format)
- Objective: goal + schema → raw JSON inputs (extraction, not derivation)
Honest boundary
Because it is fine-tuned only on flow/culture instances, it is
in-distribution on the reading and plate-culture domains and
out-of-distribution on 3D-spheroid and perfused-PK. In the benchmark:
Table with columns: domain, usable rate, self-consistent, hallucination| domain | usable rate | self-consistent | hallucination |
|---|
| 24-reading (in-dist) | 79 % | 92 % | 0.083 |
| 14 plate-culture (in-dist) | 64 % | 100 % | 0.000 |
| 15 blind (mix) | 7 % | 80 % | 0.200 |
| 15 3D-spheroid (OOD) | 0 % | 33 % | 0.611 |
| 14 perfused-PK (OOD) | 0 % | 29 % | 0.714 |
Usable = self-consistent and within ±5 % of every gold target. The OOD
collapse is the honest boundary of a fine-tuned extractor.
Load
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype="auto")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "qgeng1465/labwright-extractor")
Usage in Labwright
python -m eval.run_finetuned_benchmark --gold eval/gold_pk.json \
--out results/eval_finetuned_pk.json \
--model Qwen/Qwen2.5-1.5B-Instruct --adapter results/extractor/lora