Examples
These examples show predictions from SEL 9B.
Maze
Select every square on the shortest path from red to green.
SWE-Gym
Select the existing line of code that needs to change to fix the reported issue.
HotpotQA
Which former tennis player played for the United States, Iván Molina or Alex O’Brien? Select the supporting sentences.
Game
What can I use to climb?
Sources include SWE-Gym and Moto, HotpotQA (CC BY-SA 4.0), and LIMBO by Playdead.
Interactive examples
Serve with vLLM
Get serve.py and example.py from the SEL quickstart
and install its dependencies. Then run:
python serve.py spock-oss/sel-qwen-9b --fp8
The server downloads the pinned base, adapter, and selection head. In a
second terminal, run python example.py to try text, image, and mixed requests.
The server returns scores with source character offsets or image boxes.
Inputs are limited to 2,048 tokens, including the question and visual patches.
The command above uses FP8 on a compatible GPU. BF16 is available by omitting the flag.
Evaluation
Table with columns: Set, Mean F1, Exact mask match| Set | Mean F1 | Exact mask match |
|---|
| Task test split (3,136 examples) | 77.6% task-macro | 1,550/3,136 |
| Original 26-task gallery | 75.3% | 15/26 |
| Frozen 100 problems | 55.3% | 28/100 |
| Added development sources (17 × 64 examples) | 79.9% | 569/1,088 |
The frozen benchmark used the release thresholds. The other rows used a 0.5
cutoff. The added-source results measure source-aligned selection on prepared
development samples. A problem with multiple questions was evaluated one
question per forward.
Code ·
Research article