GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
prithivMLmods
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
prithivMLmods
Model Tree
OrionLLM/OxCoder-9B
Input Modalities
Output Modalities
Supported Functionality
| Attribute | Details |
|---|---|
| Base Model | OrionLLM/OxCoder-9B (Foundation: Qwen/Qwen3.5-9B) |
| Quantized Model | prithivMLmods/OxCoder-9B-fp8 |
| Quantization Method | LLM Compressor |
| Quantization Scheme | FP8_DYNAMIC |
| Output Format | compressed-tensors |
| Native Context Length | 262,144 tokens (262K) |
| License | Apache-2.0 |
The model was quantized to FP8 using llm-compressor with dynamic per-tensor activation scaling. Sensitive architecture components—including the language model head, input embeddings, vision modules, and linear attention layers—were excluded to preserve fidelity, reasoning stability, and code generation precision.
yaml
default_stage: default_modifiers: QuantizationModifier: targets: [Linear] ignore: - 're:.*lm_head' - 're:.*embed_tokens$' - 're:.*visual.*' - 're:.*model.visual.*' - 're:.*linear_attn.*' scheme: FP8_DYNAMIC bypass_divisibility_checks: false requires_calibration_data: falseLinear projections (MLP and attention projections).lm_head, embed_tokens, visual, model.visual, and linear_attn.compressed-tensors FP8 checkpoints run natively in vLLM:
bash
pip install vllmLaunch an OpenAI-compatible API server:
bash
vllm serve prithivMLmods/OxCoder-9B-fp8 \ --max-model-len 65536 \ --trust-remote-codeOr execute via Python:
python
from vllm import LLM, SamplingParams sampling_params = SamplingParams( temperature=0.6, top_p=0.95, max_tokens=4096) llm = LLM( model="prithivMLmods/OxCoder-9B-fp8", trust_remote_code=True, max_model_len=65536) prompts = [ "Write a Python script using asyncio to run a rate-limited web scraper with exponential backoff."] outputs = llm.generate(prompts, sampling_params)for output in outputs: print(output.outputs[0].text)bash
pip install transformers compressed-tensors acceleratepython
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "prithivMLmods/OxCoder-9B-fp8" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", torch_dtype="auto", trust_remote_code=True) prompt = "Implement a LRU cache with O(1) runtime for get and put operations in Python."inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): output_tokens = model.generate( **inputs, max_new_tokens=1024, temperature=0.6, top_p=0.95, do_sample=True ) print(tokenizer.decode(output_tokens[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))Qwen/Qwen3.5-9B.prithivMLmods.vllm serve prithivMLmods/OxCoder-9B-fp8 \ --max-model-len 65536 \ --trust-remote-codefrom vllm import LLM, SamplingParams sampling_params = SamplingParams( temperature=0.6, top_p=0.95, max_tokens=4096) llm = LLM( model="prithivMLmods/OxCoder-9B-fp8", trust_remote_code=True, max_model_len=65536) prompts = [ "Write a Python script using asyncio to run a rate-limited web scraper with exponential backoff."] outputs = llm.generate(prompts, sampling_params)for output in outputs: print(output.outputs[0].text)import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "prithivMLmods/OxCoder-9B-fp8" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", torch_dtype="auto", trust_remote_code=True) prompt = "Implement a LRU cache with O(1) runtime for get and put operations in Python."inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): output_tokens = model.generate( **inputs, max_new_tokens=1024, temperature=0.6, top_p=0.95, do_sample=True ) print(tokenizer.decode(output_tokens[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))default_stage: default_modifiers: QuantizationModifier: targets: [Linear] ignore: - 're:.*lm_head' - 're:.*embed_tokens$' - 're:.*visual.*' - 're:.*model.visual.*' - 're:.*linear_attn.*' scheme: FP8_DYNAMIC bypass_divisibility_checks: false requires_calibration_data: false