GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
prithivMLmods
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
prithivMLmods
Model Tree
OrionLLM/OxCoder-9B
Input Modalities
Output Modalities
Supported Functionality
| Attribute | Details |
|---|---|
| Base Model | OrionLLM/OxCoder-9B (Foundation: Qwen/Qwen3.5-9B) |
| Quantized Model | prithivMLmods/OxCoder-9B-fp8 |
| Quantization Method | LLM Compressor |
| Quantization Scheme | FP8_DYNAMIC |
| Output Format | compressed-tensors |
| Native Context Length | 262,144 tokens (262K) |
| License | Apache-2.0 |
The model was quantized to FP8 using llm-compressor with dynamic per-tensor activation scaling. Sensitive architecture components—including the language model head, input embeddings, vision modules, and linear attention layers—were excluded to preserve fidelity, reasoning stability, and code generation precision.
yaml
default_stage:default_modifiers:QuantizationModifier:targets: [Linear]ignore:- 're:.*lm_head'- 're:.*embed_tokens$'- 're:.*visual.*'- 're:.*model.visual.*'- 're:.*linear_attn.*'scheme: FP8_DYNAMICbypass_divisibility_checks: falserequires_calibration_data: false
Linear projections (MLP and attention projections).lm_head, embed_tokens, visual, model.visual, and linear_attn.compressed-tensors FP8 checkpoints run natively in vLLM:
bash
pip install vllm
Launch an OpenAI-compatible API server:
bash
vllm serve prithivMLmods/OxCoder-9B-fp8 \--max-model-len 65536 \--trust-remote-code
Or execute via Python:
python
from vllm import LLM, SamplingParamssampling_params = SamplingParams(temperature=0.6,top_p=0.95,max_tokens=4096)llm = LLM(model="prithivMLmods/OxCoder-9B-fp8",trust_remote_code=True,max_model_len=65536)prompts = ["Write a Python script using asyncio to run a rate-limited web scraper with exponential backoff."]outputs = llm.generate(prompts, sampling_params)for output in outputs:print(output.outputs[0].text)
bash
pip install transformers compressed-tensors accelerate
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "prithivMLmods/OxCoder-9B-fp8"tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained(model_id,device_map="auto",torch_dtype="auto",trust_remote_code=True)prompt = "Implement a LRU cache with O(1) runtime for get and put operations in Python."inputs = tokenizer(prompt, return_tensors="pt").to(model.device)with torch.no_grad():output_tokens = model.generate(**inputs,max_new_tokens=1024,temperature=0.6,top_p=0.95,do_sample=True)print(tokenizer.decode(output_tokens[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
Qwen/Qwen3.5-9B.prithivMLmods.