ikimyaii

llama-7b-sparsegpt-80pct

Model details

Table

Base model	`openlm-research/open_llama_7b`
Method	SparseGPT
Target / achieved sparsity	80% / 79.98%
Weight correction	OBS (second-order)
Retraining	none

What is pruned: every nn.Linear in the 32 decoder layers (attention q/k/v/o and MLP gate/up/down) plus lm_head, 225 layers in total (see pruning_summary.json). Embeddings and norms stay dense.

Method

SparseGPT (Frantar & Alistarh, 2023, arXiv:2301.00774) scores each weight by its OBS saliency W[i,j]² / [H⁻¹]_jj and applies column-ordered Optimal Brain Surgeon corrections to the remaining weights in blocks of 128 columns. Hessian H = (2/N) XᵀX from calibration activations, damping 0.01.

Evaluation

Table with columns: Metric, This model, Dense
Metric	This model	Dense
WikiText-2 PPL ↓	1,103.6	8.64
ARC-Easy (acc_norm)	0.255	0.678
ARC-Challenge (acc_norm)	0.265	0.385
HellaSwag (acc_norm)	0.266	0.694
PIQA (acc_norm)	0.513	0.763
WinoGrande (acc)	0.505

Zero-shot via lm-evaluation-harness; ARC/HellaSwag/PIQA acc_norm, WinoGrande acc. Raw results: results/benchmark_llama_sparsegpt_80pct.json.

WikiText-2 PPL of all five methods at 80% (dense 8.64):

Table with columns: Method, WikiText-2 PPL ↓
Method	WikiText-2 PPL ↓
CWS	948.5
SparseGPT (this model)	1,103.6
Wanda	1,252.3
RIA	1,609.5
AWP	230.10

These are the project's results at the time the checkpoints were uploaded (2026-05-11; results/sparsity_sweep_llama.json @ 6e144ae), with dense PPL 8.64. The repository README now reports these models under a 2048-token protocol (dense 6.61); the ordering between methods is similar.

Usage

python

import torchfrom transformers import AutoModelForCausalLM, LlamaTokenizer model_id = "ikimyaii/llama-7b-sparsegpt-80pct"tokenizer = LlamaTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto") lin = model.model.layers[0].mlp.down_projprint(f"sparsity: {(lin.weight == 0).float().mean():.3f}")

Limitations

These are research checkpoints for studying pruning. They are not instruction-tuned or safety-tuned, and they inherit the base model's biases.
Weights are stored densely, with zeros at the pruned positions. Sparsity alone gives no speedup without a sparse kernel or event-driven hardware.
At 70–80% sparsity this model does not produce usable text. Zero-shot accuracy is near chance.

Sparse-LLMs-on-Loihi: pruning scripts, baselines and raw results for this checkpoint.
CWS: reference implementation of the CWS method these baselines are compared against.
Weight-Sparsity-Using-Real-Loss and EHWS: follow-up work that optimises the real next-token loss instead of layer-wise reconstruction.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ikimyaii

Model Tree

Base

openlm-research/open_llama_7b

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Model details

Table

Base model	`openlm-research/open_llama_7b`
Method	SparseGPT
Target / achieved sparsity	80% / 79.98%
Weight correction	OBS (second-order)
Retraining	none

Method

Evaluation

Table with columns: Metric, This model, Dense
Metric	This model	Dense
WikiText-2 PPL ↓	1,103.6	8.64
ARC-Easy (acc_norm)	0.255	0.678
ARC-Challenge (acc_norm)	0.265	0.385
HellaSwag (acc_norm)	0.266	0.694
PIQA (acc_norm)	0.513	0.763
WinoGrande (acc)	0.505

Zero-shot via lm-evaluation-harness; ARC/HellaSwag/PIQA acc_norm, WinoGrande acc. Raw results: results/benchmark_llama_sparsegpt_80pct.json.

WikiText-2 PPL of all five methods at 80% (dense 8.64):

Table with columns: Method, WikiText-2 PPL ↓
Method	WikiText-2 PPL ↓
CWS	948.5
SparseGPT (this model)	1,103.6
Wanda	1,252.3
RIA	1,609.5
AWP	230.10

Usage

python

import torchfrom transformers import AutoModelForCausalLM, LlamaTokenizer model_id = "ikimyaii/llama-7b-sparsegpt-80pct"tokenizer = LlamaTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto") lin = model.model.layers[0].mlp.down_projprint(f"sparsity: {(lin.weight == 0).float().mean():.3f}")

Limitations

These are research checkpoints for studying pruning. They are not instruction-tuned or safety-tuned, and they inherit the base model's biases.
Weights are stored densely, with zeros at the pruned positions. Sparsity alone gives no speedup without a sparse kernel or event-driven hardware.
At 70–80% sparsity this model does not produce usable text. Zero-shot accuracy is near chance.

Sparse-LLMs-on-Loihi: pruning scripts, baselines and raw results for this checkpoint.
CWS: reference implementation of the CWS method these baselines are compared against.
Weight-Sparsity-Using-Real-Loss and EHWS: follow-up work that optimises the real next-token loss instead of layer-wise reconstruction.

llama-7b-sparsegpt-80pct

README

Model details

Method

Evaluation

Usage

Limitations

Related repositories

Explore FriendliAI today

README

Model details

Method

Evaluation

Usage

Limitations

Related repositories