aznatkoiny

gemma-4-e4b-cannabis-fda-pilot

README

License: apache-2.0

Release status

The attached receipts deliberately retain:

text
experimental=true
automated_checks_only=true
human_review_performed=false
rights_clearance_performed=false
publish_allowed=false
production_release_allowed=false
production_use_allowed=false

publish_allowed=false is the result of this project's automated production-release gate. It means the pipeline did not approve this artifact for publication or deployment; placing a copy on a model host for research transparency does not turn that failed gate into approval. The artifact must not be represented as medically reviewed, release-ready, or suitable for patient-facing use.

Training data

The pilot dataset, fda-extractive-pilot-v1, was deterministically derived from captured FDA web pages about CBD products and foods, delta-8 THC, accidental ingestion by children, pregnancy/breastfeeding, and federal cannabis-product regulation. It has:

19 retained, unique FDA extractive answer groups from five training pages;
five project-authored prompt wrappers per answer, with answers left unchanged;
80 training rows and 15 grouped validation rows;
five untouched evaluation cases from a sixth, page-disjoint FDA source; and
three source drafts excluded before tuning.

The row count therefore overstates semantic diversity: 95 tuning rows represent only 19 unique answers. Automated checks covered hashes, group separation, overlap, prompt similarity, leakage anchors, and phone/email patterns. They are not substitutes for human content, legal, scientific, or medical review. See dataset_manifest.json.

No Leafly, Weedmaps, AllBud, Reddit, scraped customer reviews, or repackaged mirrors were used. Do not add such material without explicit rights covering collection, model training, derived weights, redistribution, and the intended use.

Training

Training ran for one epoch on an NVIDIA GB10 using 4-bit NF4 QLoRA:

Table with columns: Setting, Value
Setting	Value
Base revision	`ee0ef6023621cff504d758262d4e04895a5af4a2`
Epochs	`1`
Maximum sequence length	`512`
Train/eval batch size	`4` / `4`
Gradient accumulation	`1`
Learning rate

The tested package versions are recorded in training_run.json: Transformers 5.14.1, PEFT 0.19.1, PyTorch 2.13.0, TRL 1.9.0, bitsandbytes 0.49.2, Accelerate 1.14.0, and Datasets 5.0.0.

Evaluation: failed

The adapter scored 0.50 on five held-out, page-disjoint FDA cases and the report records passed=false. All five generations were non-empty and there were no runtime errors or rubric-critical failures, but every response failed to cover the required source concepts. The score should not be read as 50% medical accuracy: each response received credit for merely being present, while its required-concept score was zero.

Both tested categories failed:

Table with columns: Category, Cases, Mean score, Passed
Category	Cases	Mean score	Passed
Medical claims	3	0.50	No
Minors	2	0.50	No

See evaluation_report.json for prompts, responses, criteria, and exact hashes.

Loading the adapter

The example pins the base model to the exact training revision. Replace ADAPTER_ID with this repository's Hub ID (or a local clone). Gemma access may require accepting Google's terms on the hosting service.

python
import torch
from peft import PeftModel
from transformers import AutoModelForMultimodalLM, AutoProcessor

ADAPTER_ID = "aznatkoiny/gemma-4-e4b-cannabis-fda-pilot"
BASE_MODEL = "google/gemma-4-E4B-it"
BASE_REVISION = "ee0ef6023621cff504d758262d4e04895a5af4a2"

# The adapter repository includes the processor/tokenizer/chat template used in training.
processor = AutoProcessor.from_pretrained(ADAPTER_ID)
base_model = AutoModelForMultimodalLM.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, ADAPTER_ID)
model.eval()

messages = [{"role": "user", "content": "What are important limitations of CBD safety data?"}]
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    add_generation_prompt=True,
    enable_thinking=False,
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        num_beams=1,
    )

generated = output[0][inputs["input_ids"].shape[-1] :]
decoded = processor.decode(generated, skip_special_tokens=False)
print(processor.parse_response(decoded, prefix=inputs["input_ids"]))

Even for research, independently verify every factual statement against current, authoritative sources. The example is an inference demonstration, not an endorsement of the generated answer.

Included artifacts and primary hashes

Table with columns: File, Purpose, SHA-256
File	Purpose	SHA-256
`adapter_model.safetensors`	LoRA adapter weights	`f46e9446ff25989d8a6300f338c51c56823734fefd792193231bd2437e3c975f`
`adapter_config.json`	PEFT configuration	`89273cdebde38a1bd4cdb5ea5f9b0f89b6736aeedaacd1749c748790be8a3958`
`chat_template.jinja`	Exact chat template	`0a2c8073c878ab1da004bee933a998606537bbb62016310352c7285c3f01c5b5`

SHA256SUMS covers the complete staged package. Base-model weights and training_args.bin are not included.

Limitations

This adapter learned a narrow FDA safety style, not strain effects or patient experiences.
Nineteen unique answers are far too few for broad capability or reliable safety.
Prompt wrappers increase formatting variety, not evidence diversity.
Source extracts and evaluation criteria received automated checks only.
The corpus is a dated snapshot; regulations, evidence, and FDA guidance can change.
The evaluation is tiny, lexical, and not clinician-authored. It does not establish factuality, calibration, fairness, robustness, or medical safety.
Fine-tuning cannot guarantee citations, prevent hallucinations, or make an LLM appropriate for high-stakes decisions.
Cannabis effects vary with composition, dose, route, tolerance, interactions, individual physiology, and many other factors not represented here.

Future work

Complete independent human medical, scientific, data-rights, and legal review.
Add substantially more unique, source-grounded examples; do not manufacture scale by repeating prompt wrappers.
Obtain explicitly licensed or consented first-party experience data and structured, quality-controlled chemistry/COA data. Track provenance at record level and exclude identifying information.
Split by source, product, batch, and contributor to prevent near-duplicate or participant leakage.
Keep changing facts in a citation-bearing retrieval layer rather than model weights; add freshness checks and an authoritative-source fallback.
Build clinician-reviewed evaluations for contraindications, drug interactions, pregnancy, minors, impairment, emergencies, uncertainty, abstention, unsupported medical claims, and adversarial prompts.
Compare the adapter against the pinned base model, use blinded human evaluation, test calibration and subgroup behavior, and publish confidence intervals.
Repeat tuning across seeds and hyperparameters only after data quality improves, then require every production release gate to pass before deployment.

License and attribution

The adapter package is offered under Apache-2.0; see LICENSE and NOTICE. The Gemma 4 base model is not included and remains subject to its own Apache-2.0 materials and notices. FDA-derived records are documented separately as CC0/public domain candidates, subject to the qualifications in THIRD_PARTY_NOTICES.md and the fact that this pilot did not complete human rights clearance. No FDA endorsement is claimed.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

aznatkoiny

Model Tree

Base

google/gemma-4-E4B-it

Adapter

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer