Examples
These examples show predictions from SEL 9B.
Maze
Select every square on the shortest path from red to green.
SWE-Gym
Select the existing line of code that needs to change to fix the reported issue.
HotpotQA
Which former tennis player played for the United States, Iván Molina or Alex O’Brien? Select the supporting sentences.
Game
What can I use to climb?
Sources include SWE-Gym and Moto, HotpotQA (CC BY-SA 4.0), and LIMBO by Playdead.
Interactive examples
Serve with vLLM
Get serve.py and example.py from the SEL quickstart
and install its dependencies. Then run:
python serve.py spock-oss/sel-qwen-0.8b
The server downloads the pinned base, adapter, and selection head. In a
second terminal, run python example.py to try text, image, and mixed requests.
The server returns scores with source character offsets or image boxes.
Inputs are limited to 2,048 tokens, including the question and visual patches.
Evaluation
Table with columns: Evaluation, Task-macro F1, Exact mask match| Evaluation | Task-macro F1 | Exact mask match |
|---|
| Validation (2,406 examples) | 65.7% | 672/2,406 |
| Test (3,136 examples) | 63.7% | 1,049/3,136 |
The validation and test results use threshold 0.5. On the frozen 100-problem
benchmark, the model reached 8/100 exact matches and 32.6% mean F1 using
the default thresholds. The exact matches were all text examples.
See the full evaluation metrics.
The earlier BF16 adapter and its runner remain available at the
previous revision.
Code ·
Research article and mixed-input examples