Model Summary
Table | |
|---|
| Base model | Qwen/Qwen3.8-27B (dense, 27B parameters) |
| Fine-tuning data | gxx27/BioTool |
| Task | Biomedical tool selection and argument generation |
| Mode | Non-thinking (enable_thinking=False) |
| Format | Merged 16-bit weights, safetensors |
| Library | Transformers (v5 or later) |
| License | Apache-2.0 |
What the Model Is Trained to Do
The model acts as the tool-calling layer of a biomedical assistant. It reads a user question together with a list of tool schemas and returns a single, well-formed function call. The tools cover three major public databases:
Table with columns: Database, Example capabilities| Database | Example capabilities |
|---|
| NCBI | E-utilities search, fetch, link and summary; BLAST |
| UniProt | Protein entries, proteomes, taxonomy, diseases, keywords, cross-references |
| Ensembl | Gene and transcript lookup, sequences, variation, VEP, coordinate mapping, comparative genomics, regulation |
It was trained to answer with a short structured call rather than a long reasoning trace, so it should be used with thinking disabled.
Intended Uses
- Research on biomedical agents and tool use in language models
- The planning step of a bioinformatics pipeline: question, tool call, execution by your own code, results
- Prototyping natural-language interfaces to NCBI, UniProt and Ensembl
- Benchmarking and comparing biomedical tool-calling approaches
Limitations
- Not for clinical use. Outputs must not inform diagnosis or treatment.
- Single-step calls only. The model produces one tool call per request. It is not trained for multi-step tool chains and does not read tool outputs to write a final answer.
- No abstention. Every training example ends in a tool call, so the model tends to call a tool even when the question is unrelated to biomedicine or no tool applies. Filter out-of-scope questions before calling it.
- Limited to the tools it has seen. Behavior on unseen tool schemas or other databases is not guaranteed.
- Arguments can be imperfect. The model may add optional parameters that were not requested, or emit placeholder-like values (for example, the string
"null"). Validate every call against the tool schema before executing it.
- Long tool schemas. Training excluded samples longer than 3,072 tokens, so very long prompts may be handled less reliably.
- Non-thinking mode only. Reasoning ability was not trained and is not expected to improve over the base model.
Evaluation
Formal results on the held-out BioTool test split are reported in the Benchmark Results section below. In an informal check of nine hand-written biomedical queries (loaded in 4-bit), the model selected the correct tool and included the key values from the question in all nine cases. Each query was presented with the correct tool plus distractor tools. This is a smoke test, not a benchmark, and should not be read as a measure of overall accuracy.
Benchmark Results
The model was tested on the held-out BioTool test split (1,408 questions) with the official BioTool evaluation code, so the scores can be compared directly with the models in the BioTool paper. For each question the model writes one tool call, and three scores are measured:
- Exact Match: the call is identical to the reference call (same tool, same arguments).
- API Success: the call runs on the live NCBI, UniProt or Ensembl service and returns data.
- BioTool Score: the model's call and the reference call are both run, and the similarity of the two results is measured with MedCPT. A call that differs slightly but returns almost the same data still earns most of the credit.
Table with columns: Score (%), NCBI, UniProt, Ensembl, Overall| Score (%) | NCBI | UniProt | Ensembl | Overall |
|---|
| Exact Match | 44.3 | 8.5 | 42.3 | 31.7 |
| API Success | 91.6 | 100.0 | 97.6 | 96.4 |
| BioTool Score | 89.5 | 90.8 | 94.6 | 91.6 |

How to read the graph. Each bar is the overall BioTool Score (higher is better). The orange bar is this model. The hatched part of its bar shows the score it would reach if the 15 questions that were not scored live (see below) had all been perfect. Dark blue bars are small models fine-tuned on BioTool, dark gray bars are frontier models used without fine-tuning, and light gray bars are small open models without fine-tuning. Scores for these 12 models are taken from the BioTool paper.
What the results show
- The model chose the correct tool for all 1,408 test questions.
- It scores higher than all four frontier models in the paper (Claude 4.5 Sonnet 81.4, Gemini 3 Pro 79.2, GPT-5.1-Codex 56.3, GPT-5.1 55.4) and all four small models without fine-tuning.
- Overall it is ahead of 11 of the 12 models. The one it does not beat is the small Qwen3-4B model (93.6) fine-tuned on the same data. Its lead over the fine-tuned Llama3.1-8B (91.2) is narrow (0.4 points) and small enough for live-database drift to matter.
- Its Exact Match (31.7%) is more than four times that of the best frontier model in the paper (7.1%).
- It is strongest on Ensembl and is ahead of every frontier model on all three databases.
Notes on these numbers
- The model was run as a 3-bit GGUF version (Q3_K_M) on a single 16 GB GPU, with greedy decoding and a single run. The paper's results are averages of three runs, and the 16-bit weights may score slightly differently.
- 15 of the 1,408 questions (1.1%) are long BLAST sequence searches. The live NCBI service was too slow to finish them in the time allowed, so they are counted as failures. The overall BioTool Score of 91.6 is therefore a lower bound, and the true value lies between 91.6 and 92.7. On another 22 BLAST questions the reference call itself failed on the live service, so no model can earn similarity credit there. Exact Match is not affected.
- After the first run, the way the model's text values (for example
null and numbers) are turned back into the right type was corrected, and the 325 affected samples were re-checked. The scores above include that re-check.
- Results from live databases can change a little over time.
Requires transformers>=5. Load the model, pass your tools in OpenAI-style schema, and disable thinking through the chat template.
# pip install -U "transformers>=5" accelerateimport re, json, torchfrom transformers import AutoTokenizer, AutoModelForMultimodalLM model_id = "Rumiii/Qwen3.8-27B-ScientificBM"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForMultimodalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto") tools = [{ "type": "function", "function": { "name": "map_cdna_to_genome", "description": "Convert from cDNA coordinates to genomic coordinates.", "parameters": { "type": "object", "properties": { "id": {"type": "string", "description": "An Ensembl stable ID"}, "region": {"type": "string", "description": "Query region"}, "include_original_region": {"type": "boolean", "description": "Include original input region"}, "species": {"type": "string", "description": "Species name/alias"}, }, "required": ["id", "region"], }, },}] messages = [ {"role": "system", "content": "You are a helpful assistant that can call tools. Use the provided tools when appropriate."}, {"role": "user", "content": "Map bases 100 to 200 of transcript ENST00000361390 to genomic coordinates in human."},] inputs = tok.apply_chat_template( messages, tools=tools, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True,).to(model.device) out = model.generate(**inputs, max_new_tokens=256, do_sample=False)text = tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)print(text) def parse_call(s): m = re.search(r"<function=([^>]+)>(.*?)</function>", s, re.S) if not m: return None args = dict(re.findall(r"<parameter=([^>]+)>\s*(.*?)\s*</parameter>", m.group(2), re.S)) return {"name": m.group(1), "arguments": args} print(json.dumps(parse_call(text), indent=2))
The model needs roughly 55 GB of memory in 16-bit. On a smaller GPU it can be loaded with 4-bit quantization, with a small expected change in output quality.
Tool calls use the Qwen3.8 chat-template format:
<tool_call><function=map_cdna_to_genome><parameter=id>ENST00000361390</parameter><parameter=region>100..200</parameter><parameter=species>human</parameter></function></tool_call>
Parsed argument values are returned as strings. Convert them to the types declared in your tool schema (for example booleans and integers) before calling the real API.
Training Data
The model was fine-tuned on BioTool, a biomedical tool-calling dataset of 7,040 query and API-call pairs across 127 tools from NCBI, UniProt and Ensembl. Only the training split was used. The test split was held out. LoRA adapters were trained and merged into the full 16-bit weights.
Acknowledgments
- Base model: Qwen/Qwen3.8-27B, licensed under Apache-2.0.
- Dataset: BioTool by Gao et al. The underlying API responses are subject to the licenses of the original NCBI, UniProt and Ensembl services.
Citation
If you use this model, please also cite the BioTool dataset:
@misc{gao2026biotoolcomprehensivetoolcallingdataset, title={BioTool: A Comprehensive Tool-Calling Dataset for Enhancing Biomedical Capabilities of Large Language Models}, author={Xin Gao and Ruiyi Zhang and Meixi Du and Peijia Qin and Pengtao Xie}, year={2026}, eprint={2605.05758}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2605.05758},}
Disclaimer
This model is provided for research purposes without warranty of any kind. Outputs may be incorrect or incomplete. Verify all tool calls and results independently before relying on them.