Read this first — the exit gate is missing
ALoDLM's adaptive inference needs a trained exit_gate.pt checkpoint to decide per-position loop
depth. That file does not exist. It was never pulled from the training box before the machine
was deleted, and it is not in this repository. This is the most important caveat on this page.
Consequences:
- Adaptive depth does not work as published. Any evaluation of this adapter must run with a
never-firing gate (
--never-exit), which forces every position through the full max_depth
iterations and reads the readout at the final depth. That is what the measurement below does.
- Do not read this repository as an adaptive-inference release. It is a weight adapter for the
trunk and the looped range; the depth-selection head is absent.
What this repository contains
Table with columns: File, What it is| File | What it is |
|---|
adapter_model.safetensors | The LoRA weights — rank 16, alpha 32, 800 tensors (435,287,728 bytes). |
adapter_config.json | PEFT configuration (PEFT 0.21.2). auto_mapping.base_model_class is ALoDLM and auto_mapping.parent_library is alodlm.model; base_model_name_or_path is null. |
alodlm_config.json | The ALoDLM loop geometry: loop_start 24, loop_end 40, max_depth 4, gate_init_pi [0.25, 0.25, 0.25, 0.25], mask_token_id 248063, block_size 4096. |
train_config.json | The training run: {"quantize": false, "precision": "bf16", "max_seq_length": 4096, "seed": 42}. |
The adapter files sit at the repository root, following the PEFT convention, so that
PeftModel.from_pretrained("Aijuus/alodlm-qwen3.8-27b-lora-1000") resolves them. alodlm_config.json
must be kept beside the adapter so the reference implementation reads the loop geometry.
What you need to use it
This is not a standalone model, and it is not a quantised radiance container. To load it you
need all of the following:
- The Qwen3.8-27B base checkpoint (
Qwen/Qwen3.8-27B).
- The ALoDLM reference implementation from the ai-setups project, under
aijuus/looping/ALoDLM/alodlm/. Plain Transformers will not build the looped trunk; the ALoDLM
class in that package does.
- This adapter's files, with
alodlm_config.json present at the same level.
adapter_config.json sets base_model_name_or_path to null, so PEFT cannot resolve a base from
this repository alone: there is no repository id to fetch. Its auto_mapping names
parent_library: alodlm.model, so a plain PeftModel.from_pretrained("Aijuus/alodlm-qwen3.8-27b-lora-1000")
in an environment without that package fails on the import before it ever looks for weights. Pass the
base checkpoint explicitly and have the ALoDLM package importable — the three requirements listed above
are all mandatory, not alternatives. Adaptive depth additionally needs an exit_gate.pt that does not
exist here (see the first section); --never-exit scoring is the only supported evaluation.
Base model availability. Qwen/Qwen3.8-27B resolves on the Hub as a public, ungated repository
under Apache-2.0, revision 37561816e66b, 32 files (checked 2026-11-08). The training run fetched
exactly that repository id, so the adapter was trained against the weights a reader can download.
Training
- Steps: 1000 optimiser steps.
- Tokens: about 1,193,500 (1,193.5 tokens per step at the corpus mean).
- Corpus: 9,900 records of generic prose, roughly 14.8 M tokens.
- Budget: this is about 8 % of a single epoch over that corpus.
- Precision: bf16,
max_seq_length 4096, seed 42.
The run is therefore under-trained and not converged. Nothing here should be read as the
capability of a fully trained ALoDLM adapter; treat every number below as preliminary.
Evaluation — likelihood proxy
The one measurement available is a per-option likelihood proxy, not sample-and-extract
generation, and it is not comparable to published MMLU figures:
- Item set: 513 MMLU items — 9 per subject, all 57 subjects, seed 42.
- Scoring: per-option conditional cross-entropy, with the argmin chosen as the answer.
- Gate: never-firing (
--never-exit), so each position runs the full 4-deep loop.
Table with columns: Arm, Accuracy, 95 % Wilson interval| Arm | Accuracy | 95 % Wilson interval |
|---|
| Base (no adapter) | 71.35 % | 67.29 – 75.09 |
| Adapter | 74.85 % | 70.92 – 78.41 |
- Delta: +3.51 points.
- Discordant pairs: 82 — 50 items gained, 32 lost.
- Exact McNemar p: 0.0598.
So the direction is favourable, but it is not statistically established at conventional
thresholds, and the two arms' confidence intervals overlap heavily. Do not cite this as a
demonstrated improvement.
What is not measured
- No task-accuracy comparison isolating the effect of the loop itself (looped vs unlooped trunk)
exists yet.
- The perplexity and KL-vs-depth measurements for this adapter are still being established.
Neither a capability claim nor a "the loop helps" claim is supported by the evidence above; they are
simply not measured yet.
Out-of-scope use
- Not for production inference, not for safety-critical use, and not as a general-purpose assistant.
- Not an adaptive-depth release — the exit gate is missing (see above).
- Not a quantised radiance container; radiance cannot load this repository directly.
Licence
This adapter inherits the base model's licence: Apache-2.0. See
Qwen/Qwen3.8-27B for the base model's terms.