GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
andyx10
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
andyx10
Model Tree
Qwen/Qwen3-4B
Input Modalities
Output Modalities
Supported Functionality
| Component | Released artifact |
|---|---|
| Oracle | Root adapter_model.safetensors + adapter_config.json; rl.qwen3-4b.e2.main.s0/iter_000425 |
| Base model | Qwen/Qwen3-4B; 36 blocks, residual width 2,560 |
| Trained oracle layers | 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, zero-based |
| Adapter | LoRA rank 16, alpha 32; 504 saved LoRA tensors |
| Generation contract | lens_config.json, code/read_oracle.py |
| Frozen reconstructor | reconstructor/, AR checkpoint ex16013344, including LoRA, head and metadata |
| Scoring statistics | whitening/, 19 frozen per-layer moment files and their hash manifest |
| Training datasets | data/, including the actual 20,000-row RL training set and 512-row gate set |
The base weights must be loaded separately. The reconstructor and whitening files are needed for reconstruction scoring, not for generating a readout.
The prompt is part of the trained lens. The user message is:
text
An activation vector from layer {layer} of a language model is enclosed in activation tags: <activation>㈎</activation>. Produce distinct concepts that encode this activation, each as a '- ' bullet on its own line.add_generation_prompt=True and
enable_thinking=False.㈎ marker (token 149705, neighboring token IDs 29, 522).16000 * v / ||v||, where v is a
nonzero, unwhitened 2,560-dimensional residual direction.The layer number describes the source activation. It does not mean injecting inside that decoder block. This model uses marker-slot embedding replacement, not the block-45 residual hook used by the separate 27B reference model.
Install PyTorch, Transformers, PEFT, safetensors and huggingface_hub. Download the small inference helper and load the adapter:
python
import importlib.utilimport torchfrom huggingface_hub import hf_hub_download repo = "andyx10/oracle-lens-qwen3-4b"helper = hf_hub_download(repo, "code/read_oracle.py")spec = importlib.util.spec_from_file_location("read_oracle", helper)lens = importlib.util.module_from_spec(spec)spec.loader.exec_module(lens)model, tokenizer, config = lens.load_oracle(device="cuda") # v must be your raw Qwen3-4B activation/direction, shape [2560].# For block 18, capture its output before the next block; do not apply final RMSNorm.v = torch.load("activation.pt", map_location="cpu", weights_only=True)print(lens.read_activation(model, tokenizer, config, v, layer=18))The helper checks marker placement and adapter-key matching. It downloads only the
root adapter and configuration, not the training corpus. For reproducible use,
pass a pinned HF commit as revision to both helper download and load_oracle.
A normal text-generation prompt without vector injection is not an oracle readout.
The sequence was on-policy chat rollouts → residual/span pairs → frozen whiteners → activation reconstructor → continuation AO → NNOMP-selected bullet distillation → GRPO. AR means text → activation; AO means activation → text.
| Measurement | Recorded value | Scope |
|---|---|---|
| Frozen AR validation FVE | 0.18017 | AR validation; reconstructor/meta.json |
| Frozen AR retrieval@1 | 0.817 | Same AR checkpoint |
| Two-pass distilled student's validation CE | 1.73465 | SFT validation; 27,994 train / 1,475 validation examples |
| Main RL initial logged gate FVE | 0.1560 | 128 gate prompts, sampled decoding |
| Main RL peak logged gate FVE | 0.1804 | eval@425; used during model selection |
| Main RL last logged gate FVE | 0.1751 | eval@575; run ended at 600 updates |
These are different diagnostics, not interchangeable benchmarks. The RL gate set
was reused during training and checkpoint selection, so its peak is not an
untouched test score. Evaluation labels are zero-based loop indices, while
checkpoint directories count completed updates: eval@425 and iter_000425
are adjacent training states, not an exact checkpoint-specific re-evaluation.
The root adapter preserves the selected iter_000425 artifact used for downstream
readouts. No new GPU evaluation is implied by this release cleanup.
The policy receives normalized AR reconstructions during RL. The reward
compares reconstructed generated concepts with true residuals in the frozen
whitened metric (ridge coefficient 0.1, joint NNLS/FVE, at most four concepts).
Main settings: 8 prompts × 64 samples, LR 5e-5, KL coefficient 0.05,
128-token generation cap, one H200 for actor and reward AR.
See training/rl/rl.qwen3-4b.e2.main.s0/
for the original configuration, prep record and log.
text
adapter_config.json / adapter_model.safetensors selected RL oraclelens_config.json prompt, geometry and injection contractreconstructor/ frozen AR LoRA + heads + metadatawhitening/ frozen moment files + manifestdata/ seeds/ rollouts/ splits/ on-policy data and split provenance pairs/ captured ao_val activation/span waves ao/chat/ ao/continuation/ pools and precomputed AR banks distillation/ all rounds, candidates and assembled shards rl/ actual train/gate Parquet filestraining/ ar/ ao/ rl/ other checkpoints and run records logs/ whitener-fit/ logs and accumulated fitting statecode/read_oracle.py standalone inference helpercode/training/ curated Qwen3-4B pipeline sourceprovenance/ full path/hash map and release metadataarchive/ original code snapshot and old manifestAll pre-cleanup artifacts are retained byte-for-byte, including intermediate
checkpoints, reference adapters, candidate generations and supplemental datasets.
data/README.md describes their schemas and roles.
provenance/file-map.json maps every original path to
its current location. Historical configs may contain the original machine paths;
these are provenance, not portable launch defaults.
The previous layout is available at revision
e0a0a7e083f0059945a383888e31e90cdc13c50a (tag legacy-layout-v1). Pin that revision
when using older scripts that expect ao/, ar/, rollouts/ or whitener/ at the
root. The clean training source is in code/training/; no copy of
upstream's privately sourced trainer is redistributed here.
The separate template lens is a fitted linear word readout. Its moments are not this oracle's whitening statistics and must not be substituted for them.
Released under the Apache License 2.0. Code adapted from
agu18dec/olens_draft keeps its MIT notice in
code/UPSTREAM_LICENSE.
Compute partially provided by a generous research grant from Modal.com.
import importlib.utilimport torchfrom huggingface_hub import hf_hub_download repo = "andyx10/oracle-lens-qwen3-4b"helper = hf_hub_download(repo, "code/read_oracle.py")spec = importlib.util.spec_from_file_location("read_oracle", helper)lens = importlib.util.module_from_spec(spec)spec.loader.exec_module(lens)model, tokenizer, config = lens.load_oracle(device="cuda") # v must be your raw Qwen3-4B activation/direction, shape [2560].# For block 18, capture its output before the next block; do not apply final RMSNorm.v = torch.load("activation.pt", map_location="cpu", weights_only=True)print(lens.read_activation(model, tokenizer, config, v, layer=18))adapter_config.json / adapter_model.safetensors selected RL oraclelens_config.json prompt, geometry and injection contractreconstructor/ frozen AR LoRA + heads + metadatawhitening/ frozen moment files + manifestdata/ seeds/ rollouts/ splits/ on-policy data and split provenance pairs/ captured ao_val activation/span waves ao/chat/ ao/continuation/ pools and precomputed AR banks distillation/ all rounds, candidates and assembled shards rl/ actual train/gate Parquet filestraining/ ar/ ao/ rl/ other checkpoints and run records logs/ whitener-fit/ logs and accumulated fitting statecode/read_oracle.py standalone inference helpercode/training/ curated Qwen3-4B pipeline sourceprovenance/ full path/hash map and release metadataarchive/ original code snapshot and old manifest