GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
anjohn0077
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
anjohn0077
Model Tree
meta-llama/Llama-3.1-8B
Input Modalities
Output Modalities
Supported Functionality
The raw mergekit extraction included full-rank modules_to_save tensors
(embed_tokens, lm_head, and RMSNorm layers) that vLLM's LoRA runtime does
not support. This upload contains only the pure low-rank lora_A/lora_B
weights (224 pairs: 32 layers x q/k/v/o/gate/up/down projections, bf16), with
modules_to_save: null in adapter_config.json. No resize_token_embeddings()
call is needed to load this adapter.
bash
python -m vllm.entrypoints.openai.api_server \--model meta-llama/Llama-3.1-8B \--enable-lora \--lora-modules finance=anjohn0077/NEXS-finance-lora \--port 8000 \--max-lora-rank 128 \--gpu-memory-utilization 0.85
| Variant | Accuracy |
|---|---|
| Base model | 0.4825 |
| This LoRA on base (via vLLM) | 0.4561 |
| Original full fine-tune | 0.5526 |
Evaluated with lm-evaluation-harness against a local vLLM OpenAI-compatible endpoint:
bash
lm_eval --model local-completions \--model_args model=finance,base_url=http://localhost:8000/v1/completions,tokenizer=meta-llama/Llama-3.1-8B,num_concurrent=10 \--tasks mmlu_econometrics \--output_path results/vllm_finance