GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
medelharchaoui
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
medelharchaoui
Model Tree
openai-community/gpt2-large
Input Modalities
Output Modalities
Supported Functionality
| Stage | WikiText-2 PPL | vs baseline |
|---|---|---|
| GPT-2 Large (baseline) | 27.56 | 1.00× |
| + ASVD rank=640 | 153.82 | 5.58× worse |
| + this LoRA adapter | 24.53 | 0.89× (11% better) |
Inference speedup retained: 1.6× (ASVD factored weights, not LoRA, do inference).
W (5120×1280) is replaced by a rank-640 factorisation U·Σ·Vᵀ, weighted by input activation statistics before SVD. This reduces matmul cost by 1.6×.A·B are added on top of frozen compressed weights, trained for 3 epochs on WikiText-2. Only 8.85M params (1.13% of model) are trainable.python
from transformers import GPT2LMHeadModel, GPT2Tokenizerfrom peft import PeftModel# Load compressed base (you must apply ASVD first — see repo for script)# OR load the full model and merge adapter weights normallybase = GPT2LMHeadModel.from_pretrained("gpt2-large")model = PeftModel.from_pretrained(base, "elharchaoui/gpt2-large-asvd640-lora")tokenizer = GPT2Tokenizer.from_pretrained("gpt2-large")inputs = tokenizer("The history of machine learning", return_tensors="pt")out = model.generate(**inputs, max_new_tokens=50)print(tokenizer.decode(out[0]))
Note: This adapter was trained on top of ASVD-compressed weights. Loading it on an uncompressed GPT-2 Large will work (the adapter corrects for the compression error) but the inference speedup requires the factored weights. See the OptIAI repository for the full compression + inference pipeline.
gpt2-large (774M params)c_fc, c_proj layers (MLP + attention projection), 36 layersc_fc, c_projSalesforce/wikitext, wikitext-2-raw-v1), 9,343 train chunks × 256 tokens| Epoch | Time | PPL wiki |
|---|---|---|
| (post-ASVD, pre-LoRA) | 0 min | 153.82 |
| 1 | 18 min | 25.57 |
| 2 | 37 min | 24.70 |
| 3 | 55 min | 24.53 |
Recovery is fast: epoch 1 already crosses below the uncompressed baseline (27.56).
Part of the OptIAI research project investigating cheap inference-time approximations of LLM weight matrices via structured decomposition.