GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
dormantx
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
dormantx
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
python
import torch, torch.nn.functional as F, yamlfrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForCausalLM, AutoTokenizerrepo = "dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-av"meta = yaml.safe_load(open(hf_hub_download(repo, "nla_meta.yaml")))tok = AutoTokenizer.from_pretrained(repo)av = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16).cuda().eval()prompt = meta["prompt_templates"]["av"].format(injection_char=meta["tokens"]["injection_char"])ids = tok(prompt, add_special_tokens=False)["input_ids"]slot = ids.index(meta["tokens"]["injection_token_id"])emb = av.get_input_embeddings()(torch.tensor(ids).cuda()[None]).clone()act = ... # a raw layer-18 residual-stream activation, i.e. hidden_states[18], shape [1536]emb[:, slot] = F.normalize(act, dim=-1).to(emb.dtype) * meta["extraction"]["injection_scale"]out = av.generate(inputs_embeds=emb,attention_mask=torch.ones(emb.shape[:2], device=emb.device),max_new_tokens=32, do_sample=False)print(tok.decode(out[0], skip_special_tokens=True))
Activations must come from hidden_states[18] of the base model (output of block 18, before the
final norm) and be passed raw — the injection step does the normalising and rescaling.
Pipeline and honesty-check methodology build on kitft/natural_language_autoencoders, kameshkanna/nla-train, and SolshineCode/nla-gemma-4-e2b.