bloomee-app

bloomee-v1-clm-nasasmd-granite4.1-3b

README

License: apache-2.0

🌍 About

This is the merged model: the QLoRA adapter bloomee-v1-clm-nasasmd-granite4.1-3b-adapter folded into ibm-granite/granite-4.1-3b and saved as ordinary fp16 weights. Load it with plain transformers — no peft, no separate base-model download.

It specialises Granite for the Bloomee agent: given a question about whether flowers are blooming somewhere, decide which NDVI tool to call and with what arguments, then report the result plainly.

It was distilled from a larger teacher (granite-4.1-8b) driven through the production agent graph — the same retrieval node, the same four tools, the same prompt — so what it learned is how the application actually behaves rather than a reimplementation of it.

If you are fine-tuning further, start from the adapter, not from here. Merging a 4-bit-trained adapter into fp16 weights is mildly lossy; the adapter is the exact artifact and this is the convenient one.

🚀 Usage

bash
pip install torch transformers
python inference.py

python
import json, torch
from pathlib import Path
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "bloomee-app/bloomee-v1-clm-nasasmd-granite4.1-3b"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.bfloat16, device_map="auto")

tools = json.loads(Path("tools.json").read_text())
messages = [
    {"role": "system", "content": Path("system_prompt.txt").read_text()},
    {"role": "user", "content": "What is the NDVI for japan_cherry today?"},
]
text = tokenizer.apply_chat_template(
    messages, tools=tools, tokenize=False, add_generation_prompt=True
)

⚠️ Three things that are not optional

Table with columns: Why
	Why
Pass `tools=`	Granite's chat template renders the four schemas into a ~400-token block inside the system message. Every training row contained it. Omit it and the model sees a prompt shape it has never encountered.
Use `system_prompt.txt` verbatim	It is the trained prompt, byte for byte. A paraphrase is a different prompt. `inference.py` checks its SHA-256 and refuses to run if it drifts.
Decode greedily	The model was evaluated at temperature 0. Sampling makes tool arguments non-reproducible.

The four tools it knows, by exact name:

markdown
current_time
list_regions_regions_get
predict_ndvi_predict__region__get
forecast_ndvi_predict__region__forecast_get

Tool calls come back Hermes-style, as <tool_call>{"name": ..., "arguments": {...}}</tool_call>. Under vLLM, serve with --enable-auto-tool-choice --tool-call-parser hermes — that is exactly how the golden scores below were produced.

📊 Results

Measured against the base model on the same harness, same frozen clock, same replayed upstream fixtures. Train/serve prompt parity was verified token-identical (1266 vs 1266 tokens), so both models were scored on prompts they were actually trained to see.

The last column matters. Only the golden conversations were run against these merged fp16 weights. The loss and generation probes were measured during training, on the base model with the adapter attached in bf16. Reporting all four as properties of this file would claim more than was checked.

Table with columns: Metric, Base, Tuned, Measured on
Metric	Base	Tuned		Measured on
Golden conversations (15 held-out)	13/15	11/15	▼	these weights, served under vLLM
Eval loss (101 held-out rows)	0.5521	0.1884	−66%	base + adapter, bf16
First tool correct (41 tool rows)	26/41 · 63%	40/41 · 98%	▲

Eval loss fell monotonically at every checkpoint, and final train loss (0.2280) sits above final eval loss — no overfitting signature at 2 epochs:

Table with columns: step, 50, 100, 150, 200, 250, 300, 350, 400
step	50	100	150	200	250	300	350	400
eval loss	0.3592	0.2496	0.2372	0.2356	0.1958	0.1948	0.1914	0.1889

A fourth number, tool-set exact (12/48 → 18/48), appears in static_eval.json. It compares a single 192-token generation against a whole multi-call tool trajectory, so it fails by construction on most rows for every model. It is kept because it is the number the original notebook reported, not because it measures anything useful.

🧠 Training

Table

Method	QLoRA, 4-bit NF4 with double quantisation, then merged to fp16
LoRA	r=16, α=32, dropout 0.05, 7 target modules (`q,k,v,o,gate,up,down_proj`)
Schedule	2 epochs, 448 steps, lr 1e-4 cosine, warmup 3%, `max_grad_norm` 0.3
Batch	1 × 8 gradient accumulation = effective 8
Precision	bf16 on A100 (sm_80), `sdpa` attention
Sequence	4096 max; supervised tokens are of each sequence

Only the final assistant turn is unmasked, and only its spans. A row is ~2,700 tokens of system prompt and retrieved context against ~60 tokens of assistant output, so training unmasked would spend most of the gradient learning to reproduce context the model is handed at inference.

📁 Files

Table

`model-0000{1..4}-of-00004.safetensors`	362 tensors, 6.35 GB, fp16, 40 layers, vocab 100352
`merged_from.json`	Provenance — the SHA-256 of the adapter these weights were merged from
`system_prompt.txt`, `tools.json`	The trained prompt and the four tool schemas
`inference.py`	Load, prompt, and parse tool calls; checks the prompt hash and prints provenance
`training_meta.json`, `static_eval.json`

merged_from.json exists because two merges of the same base model are byte-identical in shape and shard size — during this project a stale merge of a previous fine-tune was silently restored from backup and scored as if it were the new one, and nothing in a directory listing gave it away. The marker is the only tell. inference.py prints it on startup.

🛰️ Provenance

Trained on bloomee-app/bloomee-sft-nasasmd-grounded-5m — a 1,899-row synthetic corpus generated by running the production agent graph against recorded NDVI fixtures, sampling the teacher several times per seed and keeping only samples that survived validation (39.4% acceptance). That dataset card documents the schema, the intent mix and the source attributions in full.

Table with columns: Role, Model
Role	Model
Teacher	`ibm-granite/granite-4.1-8b`
Student (this model)	`ibm-granite/granite-4.1-3b`
Retrieval encoder	`ibm-granite/granite-embedding-english-r2`
Grounding scorer

The nasasmd in the name refers to NASA's Science Mission Directorate sentence transformer, which scored how well each candidate answer was grounded in the chunks actually retrieved. It is deliberately not the retrieval encoder: one model both selecting the chunks and judging groundedness would measure self-agreement rather than grounding. It shaped which samples survived into the corpus; it is not part of this model.

83 of the 1,899 rows come from nasa-impact/nasa-smd-qa-benchmark, supplying Earth-science register.

Intent mix: single_date 488 · clarification 345 · followup 224 · comparison 161 · conceptual 160 · forecast 158 · upstream_outage 95 · multilingual 72 · ambiguous 61 · injection 52 · catalogue 42 · invalid_region 41.

⚠️ Limitations

The golden-conversation score went down: 13/15 → 11/15. This is reported as measured.

All four tuned failures are extra tool calls, not wrong ones — the model learned to consult the region catalogue (and occasionally the clock) before predicting:

Table with columns: Scenario, Called, Expected
Scenario	Called	Expected
GC-04	forecast + `list_regions`	forecast
GC-07	`current_time` + list_regions + predict	list_regions + predict
GC-14	`list_regions` + predict	predict

That behaviour is what the corpus teaches — region codes are meant to be validated against the live catalogue rather than taken from model output — but the golden scenarios assert exact tool sets, so a defensible extra call scores the same as an error. GC-08 fails for both base and tuned (it states an NDVI value where the scenario forbids one); the fine-tune did not fix it.

So: tool selection improved sharply, tool economy got worse. If redundant catalogue calls matter for your use case, treat this model as a starting point rather than a finished artifact.

Other constraints:

These weights are a lossy convenience. The adapter was trained under 4-bit NF4 and merged into fp16; small numerical differences from base+adapter inference are expected. Use the adapter repo for further training, or when you need the exact trained artifact.
Eight regions only. japan_cherry, usa_cherry_dc, netherlands_tulips, france_lavender, uk_bluebells, california_poppies, texas_bluebonnets, bandung_floriculture. Anything else should be refused, and that refusal path is the corpus's thinnest intent (41 rows).
The prompt is still required. This taught behaviour, not knowledge — retrieval must still supply the context block and the tools must still be bound.
Dates were frozen to 2026-07-30 during generation. The model can invent dates from its own training prior; the tool wrapper is expected to default and clamp them.

📄 License

Apache-2.0, matching the base model.

🌟 Acknowledgments

NASA Earth Science Division for the Earth observation data behind Bloomee
NASA Space Apps Challenge for the challenge this was built for
NASA IMPACT for nasa-smd-ibm-st-v2 and the SMD QA benchmark
IBM Granite for the base and teacher models

🌸 Bloomee — Unveiling Earth's Flowering Rhythms Through Space and Time 🌸

Built with ❤️ by Team Ganespace for NASA Space Apps Challenge 2025

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

bloomee-app

Model Tree

Base

ibm-granite/granite-4.1-3b

Fine-tuned

this model

Input Modalities

Text

Output Modalities