GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
MirilAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
MirilAI
Model Tree
google/gemma-4-E2B-it
Input Modalities
Output Modalities
Supported Functionality
The paper explains the complete engineering path behind this release:
caption, answer, location, or pointing, and how downstream software validates and uses each response;It also documents the model's safety boundary and the next research steps: direct training on paired spoken questions, larger Gemma variants, stronger spatial grounding, and higher-resolution visual backbones.
These are deterministic release-checkpoint MLX 8-bit outputs on attributed Pexels images outside the training corpus. Each panel shows the ordinary user question, the parsed response, and any drawable model coordinate. The examples were manually screened for presentation and are not an accuracy sample; the complete held-out results appear below.
The user does not choose a schema or append formatting instructions. The user asks a normal question:
text
What am I looking at?
text
How many light vehicles are visible?
text
Choose a place to lower this parcel.
text
Track the white car.
The application supplies the fixed system contract from router_contract.py. The model interprets the requested operation and returns exactly one bare JSON object.
The public runtime contract is typed_json_router_v2. Pin that contract file with the model revision when integrating the model into another application.
| Requested result | JSON type | Secondary key | Downstream use |
|---|---|---|---|
| General read of the whole image | caption | none | Display or index a scene description |
| Fact, count, presence, appearance, hazard, or verbal explanation | answer | none | Display or consume a textual answer |
| Choose one landing or delivery area | location | or |
This separation is the point of the release. The response identifies its own type, so downstream code can dispatch deterministically instead of trying to classify free-form prose after generation.
Prompt for the result you need, not for the JSON structure:
| What you want | Natural examples | Response |
|---|---|---|
| A read of the complete frame | “What am I looking at?”, “Describe the whole scene.” | caption |
| A fact or explanation | “Any buses?”, “How many people?”, “What should I avoid here?” | answer |
| A candidate landing or delivery area | “Where could I land?”, “Pick a clear place for this parcel.” | location |
| One visible target | “Where is the white car?”, “Mark the storage tank.”, “Track that truck.” |
These are intent families, not a keyword list. For example, “Where are the cars concentrated?” asks for a verbal answer, while “Where is the white car?” asks for one point.
captionUse a general request for the whole frame:
text
What am I looking at?
json
{"type": "caption","caption": "From above, one built-up structure anchors the upper-left corner of a tidy property, bordered by asphalt, bright grass, driveways, and trimmed greenery."}
answerUse a question whose result should be words rather than one image point. This includes counts, object presence, colors, scene type, relative position, risks, suitability, and areas to avoid.
text
Give the visible tally for utility poles.
json
{"type": "answer","answer": "Six utility poles can be seen around the scene."}
Where does not automatically mean pointing. “Where are the vehicles concentrated?” asks for a verbal description and routes to answer. “Where is the white car?” asks for one concrete image point and routes to pointing.
locationUse this only when asking the model to choose an area for landing a drone or placing a delivery. A location response may contain a precise point, a broad directional cue, or no point at all. Its status tells the application which interpretation is valid.
text
Find a usable parcel drop location, if the view contains one.
A broad directional result can look like this:
json
{"type": "location","intent": "delivery","caption": "Coarse grid direction only, not a recommended landing or delivery spot. General safest-looking region: the open bare area on the right side of the road, away from vegetation and the building.","coordinate_system": "gemma_relative_0_1000_yx","point_2d": [500, 750],"point_semantics": "representative_point","pointing_mode": "coarse_grid_direction","status": "coarse_direction","x": 750,"y": 500}
That output identifies a direction in the frame. It does not identify an approved parcel drop or landing spot.
pointingUse this when one concrete visible point is the requested result.
text
Where in the image is the center of the white-roofed structure?
json
{"type": "pointing","action": "locate","caption": "The white-roofed structure is centered in the lower-right.","coordinate_system": "gemma_relative_0_1000_yx","point_2d": [936, 956],"point_semantics": "specific_point","pointing_mode": "precise_point","status": "target_found","x": 956,"y": 936}
The action follows the user’s intent:
| User intent | Typical wording | action |
|---|---|---|
| Find or look at one target | “Where is…”, “Find…”, “Look at…” | locate |
| Put a marker on one target | “Point at…”, “Mark…”, “Highlight…” | point |
| Supply the first point for temporal tracking | “Track…”, “Follow…”, “Lock on…” | track |
The model provides only the initial image point for track. Your application performs tracking across later video frames. The included live demo starts from a 60 x 60 region in the processed model frame, maps that region back to source-video pixels, and removes the marker if optical flow fails or any edge of the tracked box touches the frame.
Spatial responses use gemma_relative_0_1000_yx:
x=0 is the left edge and x=1000 is the right edge.y=0 is the top edge and y=1000 is the bottom edge.point_2d is [y, x].x and y repeat the same values for simpler downstream access.There are three visual outcomes. They are deliberately different because a broad cue, a grounded point, and an abstention must not look interchangeable in an operator interface.
pointing_mode: precise_point is drawable only when the response is schema-valid and uses status: target_found. It represents one image-grounded point. A UI may draw a compact reticle there. For a track request it may also initialize a tracker.
pointing_mode: coarse_grid_direction is allowed only for location. It uses a 3 x 3 grid where both coordinates are exactly 250, 500, or 750.
Treat it as “look in this part of the frame,” not “land here.” Draw a broad region cue, never a precision reticle. Do not initialize a tracker or send it to a controller as a target.
pointing_mode: none, invalid JSON, an invalid schema, or a non-target status means there is no drawable point. Coordinate fields must be null. If a malformed response combines a non-target status with coordinates, ignore those coordinates.
Interpret status by response type:
| Response | Status | Meaning |
|---|---|---|
location | target_found | One image-grounded landing or delivery candidate was selected |
location | coarse_direction | Broad 3 x 3 directional context only |
location | no_safe_area | No usable area should be marked |
The included validator enforces these rules before returning a dispatch-safe payload. Applications should still retain the original model text for audit rather than silently repairing it.
The fixed router applies this precedence:
location.pointing.answer.caption.If several indistinguishable targets are named, the model should preserve the requested pointing action but return ambiguous_target with null coordinates instead of choosing arbitrarily. If different pointing actions are mixed for one target, track takes priority over point, and point takes priority over locate.
Use transformers>=5.12.1. Gemma 4 E2B declares 35 language layers and 20 shared-KV layers. Layers 15 through 34 reuse key/value states, so a correct export does not contain separate k_proj, v_proj, or k_norm tensors for those shared layers. Release validation checks this layout directly and rejects both genuinely missing owner tensors and redundant shared-layer tensors.
The retained shared-KV export audit records the expected and observed tensor owners for this exact artifact.
Download the small runtime helpers from this repository, install their dependencies, and ask a plain-English question:
bash
hf download MirilAI/Miril-DroneVLM-2B-2 inference.py router_contract.py requirements.txt --local-dir miril-drone-runtimepython -m pip install -r miril-drone-runtime/requirements.txtpython miril-drone-runtime/inference.py \--image drone_frame.jpg \--prompt "Where is the white car?"
The script:
For direct integration, import TYPED_JSON_ROUTER_SYSTEM_PROMPT, typed_response_errors, and drawable_point from router_contract.py.
The inherited Gemma audio path can be exercised with the same image and router contract. This checkpoint was fine-tuned with typed questions, so use the published typed-versus-spoken benchmark to judge the modality gap rather than assuming parity. Install ffmpeg, then run:
bash
python miril-drone-runtime/inference.py \--image drone_frame.jpg \--audio spoken_question.mp3 \--audio-transcript "Where is the white car?"
--audio-transcript is optional provenance for the returned audit record. It is never supplied to the model; the user turn contains the image and decoded audio waveform only.
The same black-side-panel presentation used by the demo is included as runnable code. Labels are bold green, every response is shown beside the question that produced it, and the panel is half the image width.
bash
hf download MirilAI/Miril-DroneVLM-2B-2 \inference.py router_contract.py image_overlay.py video_overlay.py requirements.txt \--local-dir miril-drone-runtime
Image:
bash
python miril-drone-runtime/image_overlay.py \--image drone_frame.jpg \--output overlay.jpg \--prompt "What am I looking at?" \--prompt "Are any people or light vehicles visible?" \--prompt "Where is the white car?"
Video, with one inference pass every five seconds:
bash
python miril-drone-runtime/video_overlay.py \--input drone_video.mp4 \--output drone_video_overlay.mp4 \--interval-seconds 5 \--crop center-square \--crop-size 1000 \--prompt "What am I looking at?" \--prompt "Track the white car."
The renderer preserves the source frame rate, reveals each answer over 12 frames without changing its font size during the reveal, tracks valid precise points between inference calls, hides failed or edge-adjacent tracks, and never tracks a coarse direction or non-target response.
Release evaluation has three complementary scopes:
All scopes check route selection, strict JSON, schema validity, text overlap, status and mode agreement, null-coordinate discipline, coarse-grid agreement, and precise-coordinate recovery. No single aggregate can show all of those behaviors, so the release reports them separately. Publication requires complete prediction coverage and the exact case-file hash for every reported artifact.
| Metric | V1 merged BF16 | Merged BF16 |
|---|---|---|
| Valid JSON | 92.0% | 100.0% |
| Schema valid | 21.8% | 96.1% |
| Route accuracy | 21.8% | 94.8% |
| Caption / answer F1 | 26.2% | 38.2% |
| Spatial status | 2.9% | 79.2% |
| Precise target retained | 7.5% |
| Metric | Merged BF16 | CUDA bnb8 | CUDA bnb4 | MLX 8-bit | MLX 4-bit |
|---|---|---|---|---|---|
| Valid JSON | 100.0% | 99.1% | 96.7% | 100.0% | 100.0% |
| Schema valid | 96.1% | 93.6% | 92.1% | 94.6% | 55.8% |
| Route accuracy | 94.8% |
| Metric | Merged BF16 | CUDA bnb8 | CUDA bnb4 | MLX 8-bit | MLX 4-bit |
|---|---|---|---|---|---|
| Valid JSON | 99.9% | 99.7% | 99.0% | 100.0% | 100.0% |
| Schema valid | 99.9% | 99.7% | 98.7% | 100.0% | 53.0% |
| Route accuracy | 99.9% |
After training, a stricter held-out audit removed pointing rows whose targets fall below the model-visible size threshold, then ran every release artifact on the complete revised validation and test splits. Strict-cleaned rows use only accepted evidence. Coverage-matched rows add evidence-preserving questions on the same held-out images to restore the earlier route and pointing action/status mix; they do not recreate the earlier object-class histogram.
| Metric | Merged BF16 - Strict cleaned test | CUDA bnb8 - Strict cleaned test | CUDA bnb4 - Strict cleaned test | MLX 8-bit - Strict cleaned test | MLX 4-bit - Strict cleaned test |
|---|---|---|---|---|---|
| Valid JSON | 99.8% | 99.6% | 89.3% | 100.0% | 100.0% |
| Schema valid | 99.8% | 99.6% | 89.2% | 100.0% | 62.8% |
| Route accuracy |
| Metric | Merged BF16 - Coverage-matched test | CUDA bnb8 - Coverage-matched test | CUDA bnb4 - Coverage-matched test | MLX 8-bit - Coverage-matched test | MLX 4-bit - Coverage-matched test |
|---|---|---|---|---|---|
| Valid JSON | 99.9% | 99.6% | 89.4% | 100.0% | 100.0% |
| Schema valid | 99.9% | 99.5% | 89.3% | 100.0% | 50.2% |
| Route accuracy |
| Metric | Merged BF16 - Strict cleaned validation | CUDA bnb8 - Strict cleaned validation | CUDA bnb4 - Strict cleaned validation | MLX 8-bit - Strict cleaned validation | MLX 4-bit - Strict cleaned validation |
|---|---|---|---|---|---|
| Valid JSON | 99.9% | 99.6% | 96.4% | 100.0% | 100.0% |
| Schema valid | 99.9% | 99.5% | 96.1% | 100.0% | 64.3% |
| Metric | Merged BF16 - Coverage-matched validation | CUDA bnb8 - Coverage-matched validation | CUDA bnb4 - Coverage-matched validation | MLX 8-bit - Coverage-matched validation | MLX 4-bit - Coverage-matched validation |
|---|---|---|---|---|---|
| Valid JSON | 99.9% | 99.6% | 96.3% | 100.0% | 100.0% |
| Schema valid | 99.9% | 99.5% | 96.0% | 100.0% | 53.0% |
Validation supports comparison and model selection; test is the final held-out report. These automated scores measure contract and reference agreement, not flight safety.
| Metric | Typed question | Spoken question |
|---|---|---|
| Valid JSON | 99.9% | 100.0% |
| Schema valid | 99.9% | 98.1% |
| Route accuracy | 99.9% | 97.6% |
| Caption / answer F1 | 39.3% | 39.2% |
| Spatial status | 71.3% | 65.3% |
| Precise target retained | 50.5% |
The image, ordinary-English wording, accepted JSON target, route, and case identity are matched; only the question modality changes. This checkpoint was trained on typed questions, so spoken input is an experimental zero-shot capability boundary rather than parity-tested speech support. A future generation will train directly on paired image-plus-audio questions.
The independent benchmark rows use identical cases. Higher is better. Reference-overlap and coordinate scores are regression signals, not physical-world safety evidence.
These automated metrics are regression signals, not proof of physical-world safety. Aerial reasoning and pointing still require human review and scenario-specific testing.
bibtex
@techreport{sturges2026mirildrone2b2,title = {Miril-DroneVLM-2B-2: Turning Aerial Detector Labels into a Structured Vision-Language Interface},author = {Sturges, Stephan},institution = {Miril.ai},year = {2026},url = {https://huggingface.co/MirilAI/Miril-DroneVLM-2B-2/blob/main/technical_report.pdf}}
GPT-5.5 and GPT-5.6 Sol contributed to the research, engineering, analysis, and editing. Stephan Sturges directed the work and accepts responsibility for the report and release.
Variant cards report artifact size, recommended deployment memory, and complete same-case benchmark deltas.
Treat those measured deltas as authoritative. The CUDA 8-bit and 4-bit packages use different bitsandbytes algorithms, so bit width alone does not rank their multimodal fidelity.
The MLX 4-bit artifact remains available as a measured compact variant, not as an equivalent replacement for MLX 8-bit. On the complete balanced independent benchmark it preserves valid JSON at 100.0% after rounding but drops to 55.8% exact schema validity and 56.9% route accuracy, predominantly because pointing responses use intent where the contract requires action.
The export tensor audit passes. All expected key/value-owner and vision k_norm tensors are present; later language layers intentionally reuse shared key/value states. The leading hypothesis is therefore precision loss from uniform affine 4-bit conversion of token embeddings and linear modules, not a missing tensor. The variant card contains the full evidence and operational warning.
action key, exact route, and exact schema on the same benchmark cases.Miril-DroneVLM-2B-2 is a separate VLM, but its aerial vocabulary grows from the WALDO30 overhead detector. The two systems are complementary: WALDO supplies fast detection boxes; Miril-DroneVLM-2B-2 supplies language reasoning and typed responses.
During dataset construction, WALDO annotations are treated as authoritative evidence for class presence, counts, and geometry after audit. A broad class name is not treated as proof of a narrower noun: an UPole box may contain a utility pole, antenna, dish, mast, or related fixture, and a Boat box may contain different kinds of watercraft. Narrow language is used only when reviewed image evidence supports it. This preserves the geometric value of the detector labels without asking the model to memorize invented detail.
| WALDO class | What the class covers |
|---|---|
LightVehicle | Cars, sedans, rickshaws, vans, and similar small personal road vehicles. Trucks and buses remain separate classes. |
Person | People and human figures visible from above. |
Building | Buildings and roofed structures, including sheds, shacks, pergolas, canopies, sunshades, and tents when supported by the image. |
UPole | Utility and telegraph poles, power-line fixtures, lamp posts, street fixtures, satellite dishes, antennas, and thin truss structures. |
Boat | Boats and watercraft such as ships, canoes, kayaks, and sailboats; surfboards may also occur in the source ontology. |
The VLM receives no hidden detector boxes or object-count metadata during inference. It sees only the image, fixed router system message, and user question.
This checkpoint is derived from google/gemma-4-E2B-it and distributed as merged weights. Its aerial fine-tuning used drone-view images with typed English questions and the typed JSON contract above.
The inherited Gemma architecture also accepts audio. We evaluate image-plus-spoken-question inference separately from matched typed questions and report that result as an experimental zero-shot capability, not as speech-training parity. Native temporal video understanding is not part of this release contract: the video demo samples individual frames and uses OpenCV between model calls.
Miril-DroneVLM-2B-2 is a perception and reasoning model. It is not a flight controller, certified detect-and-avoid system, landing certifier, or standalone safety system.
It can miss small people or objects, confuse similar classes, misread scale, return an incorrect answer, select an unsuitable region, or produce invalid JSON. Performance varies with altitude, compression, motion blur, weather, camera angle, and domain shift.
For physical deployment:
Good research and prototyping fits include:
Do not use the model for direct unsupervised flight control, certified safety decisions, identifying people, or safety-critical operation without independent sensing and validation.
Apache License 2.0. See LICENSE and NOTICE.
This model is derived from google/gemma-4-E2B-it. Review the upstream model documentation and applicable terms before deployment.
intent: landingdelivery| Review a selected point or broad direction |
| Locate, mark, or track one target | pointing | action: locate, point, or track | Draw a point or initialize downstream tracking |
pointinglocationunknown |
| The image does not support a reliable judgment |
pointing | target_found | One requested target was grounded |
pointing | no_target | The requested object is absent |
pointing | no_matching_target | Objects may be present, but none satisfy the requested relation |
pointing | ambiguous_target | More than one target fits and the request does not identify one |
pointing | unknown | The image does not support a reliable point |
| 50.2% |
| Point within 100 | 3.9% | 33.4% |
| Coarse direction exact | 0.0% | 35.0% |
| No-target discipline | 43.9% | 93.4% |
| 91.6% |
| 91.2% |
| 93.4% |
| 56.9% |
| Caption / answer F1 | 38.2% | 19.6% | 37.2% | 37.3% | 35.1% |
| Spatial status | 79.2% | 63.4% | 69.7% | 77.9% | 77.3% |
| Precise target retained | 50.2% | 42.4% | 53.1% | 51.2% | 46.9% |
| Point within 100 | 33.4% | 8.0% | 29.7% | 31.0% | 25.1% |
| Coarse direction exact | 35.0% | 9.3% | 30.2% | 32.1% | 27.6% |
| No-target discipline | 93.4% | 89.6% | 87.5% | 92.6% | 91.6% |
| 99.6% |
| 98.6% |
| 100.0% |
| 53.2% |
| Caption / answer F1 | 38.6% | 23.0% | 37.2% | 37.8% | 35.6% |
| Spatial status | 86.1% | 70.1% | 79.8% | 85.3% | 82.8% |
| Precise target retained | 71.7% | 60.7% | 73.6% | 71.2% | 68.8% |
| Point within 100 | 38.3% | 9.2% | 34.0% | 33.6% | 27.8% |
| Coarse direction exact | 34.4% | 11.3% | 31.2% | 33.0% | 25.8% |
| No-target discipline | 96.3% | 93.3% | 92.4% | 95.6% | 95.5% |
| 99.8% |
| 99.5% |
| 89.1% |
| 100.0% |
| 63.3% |
| Reference text F1 | 60.9% | 47.4% | 53.5% | 60.0% | 57.2% |
| Spatial status | 88.0% | 76.7% | 72.0% | 86.1% | 83.8% |
| Coordinate quality | 89.7% | 84.0% | 78.9% | 88.4% | 86.1% |
| Precise target retained | 65.2% | 52.6% | 60.3% | 64.8% | 61.3% |
| Point within 100 | 42.6% | 9.4% | 34.3% | 39.5% | 31.4% |
| No-target discipline | 96.5% | 97.1% | 85.1% | 95.1% | 93.4% |
| 99.9% |
| 99.5% |
| 89.1% |
| 100.0% |
| 50.5% |
| Reference text F1 | 64.3% | 50.3% | 56.6% | 63.4% | 60.6% |
| Spatial status | 84.2% | 64.1% | 70.8% | 82.5% | 79.0% |
| Coordinate quality | 84.9% | 77.6% | 74.7% | 83.6% | 80.9% |
| Precise target retained | 63.5% | 49.9% | 60.0% | 62.8% | 59.7% |
| Point within 100 | 42.8% | 7.8% | 34.4% | 39.2% | 31.4% |
| No-target discipline | 96.4% | 97.6% | 84.1% | 95.2% | 93.1% |
| Route accuracy |
| 99.9% |
| 99.5% |
| 96.0% |
| 100.0% |
| 64.6% |
| Reference text F1 | 59.9% | 48.1% | 56.8% | 59.0% | 56.7% |
| Spatial status | 89.6% | 80.1% | 80.2% | 88.5% | 86.3% |
| Coordinate quality | 89.8% | 84.9% | 85.5% | 88.9% | 87.4% |
| Precise target retained | 65.8% | 55.7% | 67.8% | 65.8% | 62.9% |
| Point within 100 | 39.3% | 9.6% | 34.6% | 35.0% | 28.6% |
| No-target discipline | 96.7% | 97.2% | 91.8% | 96.1% | 95.2% |
| Route accuracy | 99.9% | 99.4% | 95.9% | 100.0% | 53.2% |
| Reference text F1 | 62.6% | 50.3% | 59.4% | 61.8% | 59.5% |
| Spatial status | 84.5% | 66.7% | 76.6% | 83.5% | 80.7% |
| Coordinate quality | 84.5% | 78.7% | 80.1% | 83.4% | 81.7% |
| Precise target retained | 60.5% | 49.7% | 62.1% | 60.7% | 58.8% |
| Point within 100 | 38.4% | 8.1% | 34.0% | 33.9% | 28.0% |
| No-target discipline | 96.1% | 97.4% | 90.0% | 95.5% | 94.1% |
| 39.3% |
| Point within 100 | 36.3% | 31.9% |
| Coarse direction exact | 34.7% | 34.3% |
| No-target discipline | 88.2% | 88.2% |
Bike | Bicycles, motorbikes, mopeds, scooters, and other two-wheeled vehicles. |
Container | Storage and shipping containers. |
Truck | Trucks, lorries, transport vehicles, semitrailers, articulated trucks, flatbeds, and larger industrial road vehicles. |
Gastank | Storage tanks, including liquid-storage and cylindrical tanks. |
Digger | Diggers, excavators, tractors when explicitly supported, and other construction vehicles or equipment. |
Solarpanels | Individual solar panels and panel arrays. |
Bus | Buses and bus-like passenger vehicles. |