AvoCahDoe

qwen2-5-7b-rlmpq-conservative

Deploy Dedicated

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

README

License: apache-2.0

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "AvoCahDoe/qwen2-5-7b-rlmpq-conservative"

model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="float16")
tokenizer = AutoTokenizer.from_pretrained(repo)

Other Qwen 2.5 7B scenarios

Table
Scenario	Avg bits	Compression	WikiText-2 PPL
Aggressive	3.1429	5.0909x	9.3678
Balanced	3.3929	4.7158x	8.9305
Extreme Survival	2.4643	6.4928x	497.4791
High Fidelity	3.75	4.2667x	8.208

Grouped archive (all scenarios in one repo): AvoCahDoe/qwen2-5-7b-rlmpq

Method

Phase 3 — PPO agent assigns per-layer bit widths under the Conservative reward target.
Phase 4 — Policy replayed on real weights; WikiText-2 perplexity validates quality.
Export — Fake-quantized FP16 weights compatible with Hugging Face Transformers.

Files

Table
File	Description
`config.json`	Llama architecture + RL-MPQ metadata
`model.safetensors`	Fake-quantized weights
`rlmpq_policy.json`	Per-layer bit-width policy
`rlmpq_metrics.json`	Validation & PPL summary

Citation

bibtex
@misc{rlmpq_qwen2_5_7b_conservative_2026,
  title  = {RL-MPQ Conservative: Qwen 2.5 7B Mixed-Precision Quantization},
  author = {AvoCahDoe},
  year   = {2026},
  url    = {https://huggingface.co/AvoCahDoe/qwen2-5-7b-rlmpq-conservative}
}