AvoCahDoe

mistral-7b-rlmpq-conservative

Deploy Dedicated

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

README

License: apache-2.0

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "AvoCahDoe/mistral-7b-rlmpq-conservative"

model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="float16")
tokenizer = AutoTokenizer.from_pretrained(repo)

Other Mistral 7B scenarios

Table
Scenario	Avg bits	Compression	WikiText-2 PPL
Aggressive	2.8125	5.6889x	10.025
Balanced	3.3438	4.785x	5.3053
Extreme Survival	2.5625	6.2439x	46.2801
High Fidelity	4.4375	3.6056x	4.8901

Grouped archive (all scenarios in one repo): AvoCahDoe/mistral-7b-rlmpq

Method

Phase 3 — PPO agent assigns per-layer bit widths under the Conservative reward target.
Phase 4 — Policy replayed on real weights; WikiText-2 perplexity validates quality.
Export — Fake-quantized FP16 weights compatible with Hugging Face Transformers.

Files

Table
File	Description
`config.json`	Llama architecture + RL-MPQ metadata
`model.safetensors`	Fake-quantized weights
`rlmpq_policy.json`	Per-layer bit-width policy
`rlmpq_metrics.json`	Validation & PPL summary

Citation

bibtex
@misc{rlmpq_mistral_7b_conservative_2026,
  title  = {RL-MPQ Conservative: Mistral 7B Mixed-Precision Quantization},
  author = {AvoCahDoe},
  year   = {2026},
  url    = {https://huggingface.co/AvoCahDoe/mistral-7b-rlmpq-conservative}
}