What it is measured to do
Evaluated on 10 SWE-bench Lite (dev) instances at 262,144 context, 60-step budget, one
run each. Metric: did the agent touch a file named in the instance's gold patch?
Table with columns: metric, result| metric | result |
|---|
| gold-file recall (located the file) | 9 / 10 (90%) |
| median tool calls to first reach it | 2 |
| mean tool calls to first reach it | 2.9 |
| episodes that MODIFIED a gold file | 7 / 10 |
| source-file writes (sed -i, redirects) | 23 |
| episodes that submitted a patch | 2 / 10 |
It edits through the shell. The evaluation scaffold exposes only a bash tool, so every
modification is a sed -i, a redirect, or a heredoc — not a structured edit call. For example,
on pvlib__pvlib-python-1606 it ran
sed -i 's/np\.Inf/np.inf/g' ./pvlib/singlediode.py, which is one of the gold-patch files.
An earlier version of this card reported "0 edit calls" and concluded the model never modifies
code. That was an instrument error: it counted calls to an edit tool the harness never
provides. The model does edit files.
Per instance:
Table with columns: instance, tool calls, first hit at| instance | tool calls | first hit at |
|---|
| marshmallow-code__marshmallow-1343 | 60 | 2 |
| marshmallow-code__marshmallow-1359 | 59 | 2 |
| pvlib__pvlib-python-1072 | 60 | 2 |
| pvlib__pvlib-python-1606 | 49 | 2 |
| pvlib__pvlib-python-1707 | 60 | 2 |
| sqlfluff__sqlfluff-1517 | 58 | 2 |
| sqlfluff__sqlfluff-1625 | 59 | 3 |
| sqlfluff__sqlfluff-1733 | 60 | never |
| sqlfluff__sqlfluff-1763 | 59 | 8 |
| sqlfluff__sqlfluff-2419 | 27 | 3 |
What it is NOT
Its editing is coarse. It modifies files with shell redirection and sed -i rather than
surgical replacements, and 3 of 10 episodes never touched a gold file. It also writes a lot of
scratch files (106 in these runs) — reproduction scripts and ad-hoc tests — so a working
directory will accumulate junk. Treat it as a capable explorer that will attempt fixes
roughly 70% of the time, not as a reliable patch generator.
Seven of ten episodes exhausted the 60-step budget without submitting. It explores long
and terminates late. Budget for that in any harness.
It was trained at 8,192 tokens but served here at 262,144. The longer window is real
(llama.cpp reports n_ctx_slot = 262144 and an independent 67,243-token needle test passed),
but quality beyond the trained length is unverified — the window prevents truncation, it does
not guarantee comprehension.
Ten instances is a small sample. 90% recall on n=10 carries wide error bars. Treat it as a
strong indication, not a headline benchmark result.
Reproduce the measurement
The evaluation used mini-swe-agent against a
local llama-server:
llama-server -m amaara-code-explore-Q8_0.gguf \ -ngl 99 -c 262144 --parallel 1 \ -ub 512 -b 1024 --jinja --host 0.0.0.0 \ --cache-type-k q8_0 --cache-type-v q8_0
--parallel 1 matters: -c is a shared pool, so with the default 4 slots a 262k setting
silently gives each request 65k.
Training
Table with columns: stage, steps, context, notes| stage | steps | context | notes |
|---|
| 1 | 3,000 | 4,096 | harness + agent traces |
| 2 | 2,000 | 8,192 | continued from stage 1; 28.2% of loss tokens from generalised Paperclip agent traces |
Stage 2 was aimed at a specific defect: the stage-1 model repeated identical tool calls in
65% of episodes. After stage 2 the duplicate rate fell to 26%, and after raising the serving
context from 32k to 98k one instance fell from 41.8% to 7.0% — part of the original figure was
truncation, not looping.
Training data was generalised before use: client names, paths, personal names and task codes
were substituted with stable pseudonyms, and 25 banned patterns were verified absent from the
final corpus.
Quants
Table with columns: file, size, notes| file | size | notes |
|---|
amaara-code-explore-F16.gguf | 8.1 GB | reference |
amaara-code-explore-Q8_0.gguf | 4.3 GB | the configuration measured above |
amaara-code-explore-Q6_K.gguf | 3.4 GB | |
amaara-code-explore-Q5_K_M.gguf | 3.0 GB | |
amaara-code-explore-Q4_K_M.gguf | 2.6 GB | smallest |
Quantisation damage has not been measured for this model. The numbers in this card were
taken on Q8_0; if you use Q4_K_M, re-measure before relying on them.
Licence
Apache-2.0, inherited from the Qwen3.5-4B base.
Join the AI Owner's Club
We build these models in the open with a community of people who want to own their AI instead
of renting it. If you're training your own models, running them on your own hardware, and refusing
to accept that you have to pay per token forever, you'll fit right in.
Join the AI Owner's Club on Skool