GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
thisisaditichouhan
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
thisisaditichouhan
Model Tree
Base
microsoft/phi-2
Adapter
this model
Input Modalities
Text
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
The original Phi-2 model runs on 16-bit precision (Float16) and consumes a lot of memory. To make it highly efficient and runnable on free-tier cloud GPUs (like Google Colab T4) or local machines with limited VRAM, this model has been compressed to 4-bit using NormalFloat4 (NF4) quantization.
microsoft/phi-2bitsandbytes)BitsAndBytesConfig with load_in_4bit=True and bnb_4bit_compute_dtype=torch.float16.LoraConfig targeting the standard query/value projection layers (q_proj, v_proj).adapter_model.safetensors) and tokenizer configuration.This repository is perfect for anyone looking to experiment with lightweight text generation or perform Parameter-Efficient Fine-Tuning (PEFT) on top of a 4-bit quantized version of Phi-2.