GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
vxltxrllc
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
vxltxrllc
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Voxtral-Mini-3B Text-Llama
vxltxrllc/voxtral-mini-3b-text-llama is a standalone, text-only extraction of the language model backbone from Mistral AI's Voxtral-Mini-3B.
By stripping away the speech encoder and cross-attention/audio adapter projections, this repository provides a lightweight (~3B parameters) pure-text decoder converted directly into standard Llama architecture (LlamaForCausalLM). Key Highlights
text
Standard Llama Compatibility: Fully compatible out-of-the-box with vLLM, llama.cpp, Ollama, SGLang, TGI, and standard transformers pipelines without custom model code or multimodal overrides. Optimized Footprint: Removes all non-text weights, resulting in faster cold starts, lower RAM/VRAM consumption, and clean quantization pipelines (GGUF, AWQ, GPTQ). Architecture: Pure Causal LM based on the Transformer architecture using Rotary Position Embeddings (RoPE), SwiGLU activations, and Grouped-Query Attention (GQA).Model Details Developed by: Extraction & conversion by vxltxr LLC && Original base weights by Mistral AI Model Type: Causal Language Model (LlamaForCausalLM) Parameter Count: ~3B Context Window: 8,192 tokens License: MIT
Quickstart & Usage
Python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM model_id = "vxltxrllc/voxtral-mini-3b-text-llama" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto") prompt = "Explain the difference between streaming ASR and offline ASR in two sentences."inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=128, temperature=0.7, top_p=0.9, do_sample=True) print(tokenizer.decode(outputs[0], skip_special_tokens=True))Because the model was mapped directly to LlamaForCausalLM, you can launch it with vLLM immediately: Bash
Bash
vllm serve vxltxrllc/voxtral-mini-3b-text-llama \ --dtype bfloat16 \ --max-model-len 8192 \ --port 8000Extraction & Conversion Methodology
The weight conversion pipeline performs the following surgical operations:
text
Filter Keys: Strips audio encoder blocks (audio_encoder.*), projector layers, and cross-attention keys. Key Mapping: Remaps internal layer naming to standard LlamaForCausalLM conventions (model.layers.{i}.self_attn.*, model.embed_tokens, lm_head). Config Synthesis: Generates a standard config.json defining architectures: ["LlamaForCausalLM"] with native RoPE theta and GQA parameters matching the extracted shapes.Intended Use & Limitations Intended Use
text
Lightweight text generation and conversational modeling. Downstream fine-tuning (LoRA, QLoRA, DPO/ORPO). Edge device deployment via llama.cpp / GGUF quantization.Limitations
text
No Audio Processing: This model cannot accept audio/speech inputs. For speech-to-text or multimodal tasks, use the original mistralai/Voxtral-Mini-3B. Inherited Biases: Retains any baseline capabilities, factual limitations, or language biases present in the original model.Citation & Acknowledgments
If you use this converted model in your work or benchmarks, please cite the original authors and the extraction repository: Code snippet
@misc{voxtral_mini_3b_text_llama_2026, author = {vxltxr LLC}, title = {Voxtral-Mini-3B Text-Llama: Standalone Llama Extraction}, year = {2026}, publisher = {Hugging Face}, journal = {Hugging Face repository}, howpublished = {\url{https://huggingface.co/vxltxrllc/voxtral-mini-3b-text-llama}} }
Standard Llama Compatibility: Fully compatible out-of-the-box with vLLM, llama.cpp, Ollama, SGLang, TGI, and standard transformers pipelines without custom model code or multimodal overrides.Optimized Footprint: Removes all non-text weights, resulting in faster cold starts, lower RAM/VRAM consumption, and clean quantization pipelines (GGUF, AWQ, GPTQ).Architecture: Pure Causal LM based on the Transformer architecture using Rotary Position Embeddings (RoPE), SwiGLU activations, and Grouped-Query Attention (GQA).import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM model_id = "vxltxrllc/voxtral-mini-3b-text-llama" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto") prompt = "Explain the difference between streaming ASR and offline ASR in two sentences."inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=128, temperature=0.7, top_p=0.9, do_sample=True) print(tokenizer.decode(outputs[0], skip_special_tokens=True))vllm serve vxltxrllc/voxtral-mini-3b-text-llama \ --dtype bfloat16 \ --max-model-len 8192 \ --port 8000Filter Keys: Strips audio encoder blocks (audio_encoder.*), projector layers, and cross-attention keys.Key Mapping: Remaps internal layer naming to standard LlamaForCausalLM conventions (model.layers.{i}.self_attn.*, model.embed_tokens, lm_head).Config Synthesis: Generates a standard config.json defining architectures: ["LlamaForCausalLM"] with native RoPE theta and GQA parameters matching the extracted shapes.Lightweight text generation and conversational modeling.Downstream fine-tuning (LoRA, QLoRA, DPO/ORPO).Edge device deployment via llama.cpp / GGUF quantization.No Audio Processing: This model cannot accept audio/speech inputs. For speech-to-text or multimodal tasks, use the original mistralai/Voxtral-Mini-3B.Inherited Biases: Retains any baseline capabilities, factual limitations, or language biases present in the original model.