Results
All models trained on the same 70,890 CORINE questions from 5% of BigEarthNet v2.0 (three seeds) and timed in one
job on one H100 PCIe GPU (batch 1, bfloat16; each timer runs from the decoded image in memory to the answer).
CORINE: 36,028 test questions (mean ± std over seeds), ECE = mean of the per-seed ECEs (15 bins). RSVQAxBEN: 29,501
presence questions written by other authors, never used for training. ms / image: all the questions of an image
(6.1 on average) in one request.
Table with columns: Model, Params, Input, CORINE acc. (%), ECE, RSVQAxBEN acc. (%), ms / question, Slower than EarthJev, ms / image, Slower than EarthJev| Model | Params | Input | CORINE acc. (%) | ECE | RSVQAxBEN acc. (%) | ms / question | Slower than EarthJev | ms / image | Slower than EarthJev |
|---|
| EarthJev (this model) | 0.8B | 12 bands | 86.0 ± 0.1 | 0.006 | 81.0 | 19.9 | — | 42.8 | — |
| Qwen3.5-0.8B, generative | 0.8B | 12 bands | 86.8 ± 0.2 | 0.005 | 80.5 | 134.7 | 6.8× | 191.9 | 4.5× |
| Qwen3-VL-2B, LoRA | 2.1B | RGB | 86.6 ± 0.1 | 0.005 | 80.6 | 111.6 | 5.6× | 176.2 | 4.1× |
| InternVL3-2B, LoRA | 2.1B | RGB | 87.1 ± 0.0 | 0.006 | 81.2 | 84.5 | 4.3× | 157.3 | 3.7× |
| EarthDial-4B, LoRA | 4.2B | RGB | 89.5 ± 0.1 | 0.006 | 82.4 | 171.8 | 8.7× | 431.6 | 10.1× |
| ResNet-50 VQA* | 0.14B | 12 bands | 87.5 ± 0.2 | 0.089 | 79.4 | 12.1 | 0.6× | 14.9 | 0.3× |
| Swin-T VQA* | 0.15B | 12 bands | 86.5 ± 0.1 | 0.047 | 79.6 | 16.2 | 0.8× | 17.9 | 0.4× |
| MambaVision-T VQA* | 0.15B | 12 bands | 85.9 ± 0.3 | 0.064 | 78.4 | 15.3 | 0.8× | 18.6 | 0.4× |
* Fixed-vocabulary VQA models: a reference for the cost of a model that answers only the questions and options it
was trained on. Replacing the nine non-RGB bands with their training mean costs EarthJev 4.5 points on CORINE.
Usage
Requirements: transformers with Qwen3.5 support, torch, and OneJev's engine
(pip install "qev @ git+https://github.com/OmniJev/OneJev"). From a clone of this repository:
import numpy as npfrom earthjev_infer import EarthJev model = EarthJev.load(".") # this folder (downloaded with huggingface_hub.snapshot_download)s2 = np.load("examples/S2A_MSIL2A_20170701T093031_N9999_R136_T35VPK_37_55.npy") # [12, 120, 120] uint16answers = model.predict(s2, { "dominant": {"type": "choice", "instructions": "Which land-cover class covers most of this patch?", "criteria": {"artificial": "Artificial surfaces.", "agricultural": "Agricultural areas.", "forest": "Forest and seminatural areas.", "wetlands": "Wetlands.", "water": "Water bodies."}}, "urban": {"type": "noul", "instructions": "Is there any urban fabric in this patch?"},})print(answers["dominant"]) # a probability for every option
The input is an array [12, H, W] with the Sentinel-2 L2A bands B01 B02 B03 B04 B05 B06 B07 B08 B8A B09 B11 B12 at
10 m (20 m and 60 m bands resampled by pixel repetition), digital numbers as distributed (reflectance × 10,000), as
in BigEarthNet v2.0. examples/ holds two test patches (7 questions, 7 questions) with
their CORINE test questions and an RGB preview.
Intended use and limitations
- Research on question answering and decision models for remote sensing. Not validated for operational use.
- Trained on 11,893 patches of 120 × 120 pixels from ten European countries (2017–2018) and on questions generated
from CORINE Land Cover (minimum mapping unit 25 ha); accuracy outside this distribution, on other sizes, products
(e.g. L1C) or question types is not established. On BigEarthNet.txt questions, whose types differ from the training
questions, accuracy is about 60% on binary and 40% on four-option questions.
- Only questions with declared options: open requests ("describe this image") are outside the model's interface.
Training data and licenses
Weights: Apache-2.0 (base model Qwen3.5-0.8B, Apache-2.0). Training and test data: BigEarthNet v2.0 (reBEN) and
BigEarthNet.txt (CDLA-Permissive-1.0; labels derived from CORINE Land Cover, © European Union, Copernicus Land
Monitoring Service); RSVQAxBEN (CC-BY-4.0) for testing. The example patches come from the reBEN test split.
Citation
@article{benfenati2026earthjev, title = {{EarthJev}: Calibrated, Non-Generative Answers to Natural-Language Questions about {Sentinel-2} Land Cover}, author = {Benfenati, Domenico and Rinaldi, Antonio Maria and Tommasino, Cristian}, journal = {arXiv preprint arXiv:2611.02096}, year = {2026}, url = {https://arxiv.org/abs/2611.02096}}
This study was partially supported by the PNRR MUR project PE0000013-FAIR.