Joesh1

onca-2.0-12b-INT4

README

License: apache-2.0

Summary

ONCA 2.0 is an open oncology language model for trial screening, clinical reasoning, pathology extraction, and variant evidence interpretation. It builds on google/gemma-4-12B-it with continued supervised fine-tuning on a provenance-labeled oncology corpus while retaining the four-task ONCA 1.5 evaluation contract.

This repository contains the merged 4-bit BitsAndBytes checkpoint, quantized from the BF16 reference release and exported on 2026-06-14. It is the lowest-memory Transformers deployment in the ONCA 2.0 Hugging Face family.

At a Glance

Table with columns: Field, Value
Field	Value
Release	4-bit BitsAndBytes release
Base model	`google/gemma-4-12B-it`
Architecture	Gemma 4 unified 12B (`Gemma4UnifiedForConditionalGeneration`)
Context window	262,144 tokens
Quantization	BitsAndBytes NF4, double quantization, BF16 compute
Domain focus	Pancreatic cancer and oncology research
Weights	Two `safetensors` shards

Table with columns: Task, Headline metric, BF16 reference
Task	Headline metric	BF16 reference
Trial Screening	Accuracy	0.8240
Clinical Reasoning	Outcome-label accuracy	0.6761
Pathology Extraction	Overall field exact match	0.4634
Variant Evidence	Clinical-significance macro-F1	0.5427

The INT4 export passed loading checks but was not independently rebenchmarked across the full holdout; quantization-related differences may occur.

Quick Start

Use recent versions of Transformers, Accelerate, and BitsAndBytes with Gemma 4 support.

python
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "Joesh1/onca-2.0-12B-INT4"

processor = AutoProcessor.from_pretrained(model_id)
tokenizer = processor.tokenizer
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

# Example: criterion-aware trial screening with structured output
messages = [{
    "role": "user",
    "content": (
        "Patient: metastatic pancreatic adenocarcinoma; ECOG 1; "
        "no prior metastatic-line therapy. Trial: metastatic PDAC, ECOG 0-1, "
        "no prior metastatic-line therapy. Return JSON with keys eligible, "
        "reason, and missing_information."
    ),
}]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=160, do_sample=False)
answer = tokenizer.decode(
    outputs[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(answer)

The saved config.json contains the NF4 quantization metadata. For structured workflows, request exact fields, provide all relevant criteria, ask for explicit uncertainty, and prefer deterministic decoding.

Training Scope

The source BF16 checkpoint was trained on 25,302 examples. Validation, test, and primary-holdout sets retain the ONCA 1.5 four-task evaluation contract.

Table with columns: Task family, Train, Original, Generated, Val, Test, Holdout
Task family	Train	Original	Generated	Val	Test	Holdout
Trial Screening	10,921	10,921	0	608	608	608
Clinical Reasoning	3,647	3,146	501	174	176	176

onca-2.0-12B: BF16 reference release.
onca-2.0-12B-INT8: 8-bit BitsAndBytes release.
onca-2.0-12B-INT4: 4-bit BitsAndBytes release (this page).
onca-2.0-12B-GGUF: llama.cpp-compatible GGUF collection.

Limitations

This is a research model, not a clinical decision system.
Outputs require review by qualified experts before real-world use.
Structured or parser-valid output does not guarantee factual correctness.
Quantization may cause behavior to differ from the BF16 reference checkpoint.
The full primary holdout was not independently rerun for this INT4 export.

Citation

A formal ONCA 2.0 citation will be added with the accompanying manuscript. Until then, cite this model repository and the exact version used.

Acknowledgements

ONCA 2.0 continues the ONCA project lineage and builds on Google Gemma and the open-data contributors whose datasets supported training and evaluation.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Joesh1

Model Tree

Base

google/gemma-4-12B-it

Quantized

this model

Input Modalities

TextAudioImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Summary

At a Glance

Table with columns: Field, Value
Field	Value
Release	4-bit BitsAndBytes release
Base model	`google/gemma-4-12B-it`
Architecture	Gemma 4 unified 12B (`Gemma4UnifiedForConditionalGeneration`)
Context window	262,144 tokens
Quantization	BitsAndBytes NF4, double quantization, BF16 compute
Domain focus	Pancreatic cancer and oncology research
Weights	Two `safetensors` shards

Table with columns: Task, Headline metric, BF16 reference
Task	Headline metric	BF16 reference
Trial Screening	Accuracy	0.8240
Clinical Reasoning	Outcome-label accuracy	0.6761
Pathology Extraction	Overall field exact match	0.4634
Variant Evidence	Clinical-significance macro-F1	0.5427

The INT4 export passed loading checks but was not independently rebenchmarked across the full holdout; quantization-related differences may occur.

Quick Start

Use recent versions of Transformers, Accelerate, and BitsAndBytes with Gemma 4 support.

python
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "Joesh1/onca-2.0-12B-INT4"

processor = AutoProcessor.from_pretrained(model_id)
tokenizer = processor.tokenizer
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

# Example: criterion-aware trial screening with structured output
messages = [{
    "role": "user",
    "content": (
        "Patient: metastatic pancreatic adenocarcinoma; ECOG 1; "
        "no prior metastatic-line therapy. Trial: metastatic PDAC, ECOG 0-1, "
        "no prior metastatic-line therapy. Return JSON with keys eligible, "
        "reason, and missing_information."
    ),
}]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=160, do_sample=False)
answer = tokenizer.decode(
    outputs[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(answer)

Training Scope

The source BF16 checkpoint was trained on 25,302 examples. Validation, test, and primary-holdout sets retain the ONCA 1.5 four-task evaluation contract.

Table with columns: Task family, Train, Original, Generated, Val, Test, Holdout
Task family	Train	Original	Generated	Val	Test	Holdout
Trial Screening	10,921	10,921	0	608	608	608
Clinical Reasoning	3,647	3,146	501	174	176	176

onca-2.0-12B: BF16 reference release.
onca-2.0-12B-INT8: 8-bit BitsAndBytes release.
onca-2.0-12B-INT4: 4-bit BitsAndBytes release (this page).
onca-2.0-12B-GGUF: llama.cpp-compatible GGUF collection.

Limitations

This is a research model, not a clinical decision system.
Outputs require review by qualified experts before real-world use.
Structured or parser-valid output does not guarantee factual correctness.
Quantization may cause behavior to differ from the BF16 reference checkpoint.
The full primary holdout was not independently rerun for this INT4 export.

Citation

A formal ONCA 2.0 citation will be added with the accompanying manuscript. Until then, cite this model repository and the exact version used.

Acknowledgements

ONCA 2.0 continues the ONCA project lineage and builds on Google Gemma and the open-data contributors whose datasets supported training and evaluation.

onca-2.0-12b-INT4

README

Summary

At a Glance

Quick Start

Training Scope

Related Releases

Limitations

Citation

Acknowledgements

Explore FriendliAI today

README

Summary

At a Glance

Quick Start

Training Scope

Related Releases

Limitations

Citation

Acknowledgements