bloomee-app

bloomee-v1-clm-nasasmd-granite4.1-3b-adapter

README

License: apache-2.0

🌍 About

This is a LoRA adapter, not a standalone model. It attaches to ibm-granite/granite-4.1-3b and specialises it for the Bloomee agent: given a question about whether flowers are blooming somewhere, decide which NDVI tool to call and with what arguments, then report the result plainly.

It was distilled from a larger teacher (granite-4.1-8b) driven through the production agent graph — the same retrieval node, the same four tools, the same prompt — so what it learned is how the application actually behaves rather than a reimplementation of it.

The adapter is 31.1 M trainable parameters (0.91% of the 3.43 B base), ~124 MB on disk.

🚀 Usage

bash
pip install torch transformers peft
python inference.py

python
import json, torch
from pathlib import Path
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "ibm-granite/granite-4.1-3b"
tokenizer = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "bloomee-v1-clm-nasasmd-granite4.1-3b-adapter")

tools = json.loads(Path("tools.json").read_text())
messages = [
    {"role": "system", "content": Path("system_prompt.txt").read_text()},
    {"role": "user", "content": "What is the NDVI for japan_cherry today?"},
]
text = tokenizer.apply_chat_template(
    messages, tools=tools, tokenize=False, add_generation_prompt=True
)

⚠️ Three things that are not optional

Table with columns: Why
	Why
Pass `tools=`	Granite's chat template renders the four schemas into a ~400-token block inside the system message. Every training row contained it. Omit it and the model sees a prompt shape it has never encountered.
Use `system_prompt.txt` verbatim	It is the trained prompt, byte for byte. A paraphrase is a different prompt. `inference.py` checks its SHA-256 and refuses to run if it drifts.
Decode greedily	The model was evaluated at temperature 0. Sampling makes tool arguments non-reproducible.

The four tools it knows, by exact name:

markdown
current_time
list_regions_regions_get
predict_ndvi_predict__region__get
forecast_ndvi_predict__region__forecast_get

Tool calls come back Hermes-style, as <tool_call>{"name": ..., "arguments": {...}}</tool_call>. Under vLLM, serve with --enable-auto-tool-choice --tool-call-parser hermes.

📊 Results

Measured against the base model on the same harness, same frozen clock, same replayed upstream fixtures. Train/serve prompt parity was verified token-identical (1266 vs 1266 tokens), so both models were scored on prompts they were actually trained to see.

Table with columns: Metric, Base, Tuned
Metric	Base	Tuned
Eval loss (101 held-out rows)	0.5521	0.1884	−66%
First tool correct (41 tool rows)	26/41 · 63%	40/41 · 98%	▲
Correctly silent (7 no-tool rows)	0/7	6/7	▲
Golden conversations (15 held-out)	13/15

Eval loss fell monotonically at every checkpoint, and final train loss (0.2280) sits above final eval loss — no overfitting signature at 2 epochs:

Table with columns: step, 50, 100, 150, 200, 250, 300, 350, 400
step	50	100	150	200	250	300	350	400
eval loss	0.3592	0.2496	0.2372	0.2356	0.1958	0.1948	0.1914	0.1889

🧠 Training

Table

Method	QLoRA, 4-bit NF4 with double quantisation
LoRA	r=16, α=32, dropout 0.05, 7 target modules (`q,k,v,o,gate,up,down_proj`)
Schedule	2 epochs, 448 steps, lr 1e-4 cosine, warmup 3%, `max_grad_norm` 0.3
Batch	1 × 8 gradient accumulation = effective 8
Precision	bf16 on A100 (sm_80), `sdpa` attention
Sequence	4096 max; supervised tokens are 2.8% of each sequence

Only the final assistant turn is unmasked, and only its spans. A row is ~2,700 tokens of system prompt and retrieved context against ~60 tokens of assistant output, so training unmasked would spend most of the gradient learning to reproduce context the model is handed at inference.

🛰️ Provenance

Trained on bloomee-app/bloomee-sft-nasasmd-grounded-5m — a 1,899-row synthetic corpus generated by running the production agent graph against recorded NDVI fixtures, sampling the teacher several times per seed and keeping only samples that survived validation (39.4% acceptance). That dataset card documents the schema, the intent mix and the source attributions in full.

Table with columns: Role, Model
Role	Model
Teacher	`ibm-granite/granite-4.1-8b`
Student (this adapter)	`ibm-granite/granite-4.1-3b`
Retrieval encoder	`ibm-granite/granite-embedding-english-r2`
Grounding scorer

The nasasmd in the name refers to NASA's Science Mission Directorate sentence transformer, which scored how well each candidate answer was grounded in the chunks actually retrieved. It is deliberately not the retrieval encoder: one model both selecting the chunks and judging groundedness would measure self-agreement rather than grounding. It shaped which samples survived into the corpus; it is not part of this adapter.

83 of the 1,899 rows come from nasa-impact/nasa-smd-qa-benchmark, supplying Earth-science register.

Intent mix: single_date 488 · clarification 345 · followup 224 · comparison 161 · conceptual 160 · forecast 158 · upstream_outage 95 · multilingual 72 · ambiguous 61 · injection 52 · catalogue 42 · invalid_region 41.

⚠️ Limitations

The golden-conversation score went down: 13/15 → 11/15. This is reported as measured.

All four tuned failures are extra tool calls, not wrong ones — the model learned to consult the region catalogue (and occasionally the clock) before predicting:

Table with columns: Scenario, Called, Expected
Scenario	Called	Expected
GC-04	forecast + `list_regions`	forecast
GC-07	`current_time` + list_regions + predict	list_regions + predict
GC-14	`list_regions` + predict	predict

That behaviour is what the corpus teaches — region codes are meant to be validated against the live catalogue rather than taken from model output — but the golden scenarios assert exact tool sets, so a defensible extra call scores the same as an error. GC-08 fails for both base and tuned (it states an NDVI value where the scenario forbids one); the fine-tune did not fix it.

So: tool selection improved sharply, tool economy got worse. If redundant catalogue calls matter for your use case, treat this adapter as a starting point rather than a finished artifact.

Other constraints:

Eight regions only. japan_cherry, usa_cherry_dc, netherlands_tulips, france_lavender, uk_bluebells, california_poppies, texas_bluebonnets, bandung_floriculture. Anything else should be refused, and that refusal path is the corpus's thinnest intent (41 rows).
The prompt is still required. This taught behaviour, not knowledge — retrieval must still supply the context block and the tools must still be bound.
Dates were frozen to 2026-07-30 during generation. The model can invent dates from its own training prior; the tool wrapper is expected to default and clamp them.
English-dominant. 72 of 1,899 rows are non-English.

📄 License

Apache-2.0, matching the base model.

🌟 Acknowledgments

NASA Earth Science Division for the Earth observation data behind Bloomee
NASA Space Apps Challenge for the challenge this was built for
NASA IMPACT for nasa-smd-ibm-st-v2 and the SMD QA benchmark
IBM Granite for the base and teacher models
Hugging Face PEFT for the adapter tooling

🌸 Bloomee — Unveiling Earth's Flowering Rhythms Through Space and Time 🌸

Built with ❤️ by Team Ganespace for NASA Space Apps Challenge 2025

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider