GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AutomatosX
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
AutomatosX
Model Tree
Qwen/Qwen3-VL-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
No quantization-container correction was needed. This family has no n-gram tensors; no n-gram file or declaration was added. This CUDA pack is outside MLX/oMLX/MTPLX export scope.
See runtime_audit.json for pinned config/index/header bindings, architecture, physical-format findings, and applied corrections. This is development evidence; no quality, MTP exactness, speed, or certification claim is added. Historical evidence stays bound to its original revision.
Native AXQuant CUDA NVFP4 W4A4 development preview. Original official BF16 language projections are converted with AXQuant RTN, without AWQ. E2M1 FP4 weights and inputs use E4M3FN scales per 16 values and FP32 global scales. The complete vision tower, DeepStack mergers, embeddings, norms and LM head retain original BF16 values, independently checked for equality.
No native MTP. This official Qwen3-VL source contains no trained MTP head. Quantization does not add one.
0c351dd01ed87e9c1b53cbc748cba10e6187ff3b.1605cc05538fba8f12a5c8af051e3999ea06e947.LICENSE.Factory conversion uses AXQuant's numpy-reference weight encoder.
Inference is tested separately on NVIDIA GPUs. This is an unreleased
development export; older published AXQuant wheels do not contain this path.
Exact code, source-shard and calibration hashes are in provenance.json.
Calibration observes original BF16 inputs for every selected matrix, binds source checksums, column shapes, sample counts and the test image, and uses 1.25 activation range headroom. Fused Q/K/V and gate/up share global scales. Input block scales are computed dynamically during inference.
Official vLLM 0.25.1, PyTorch 2.11.0+cu130, CUDA 13.0, on both
RTX 5090 and Jetson Thor. Each worker reports 144 native
CutlassNvFp4LinearKernel modules; there are no MoE tables in these dense
models. Chunked prefill is enabled. Both GPUs recognize all three test-page
lines, and the check rejects repeated nonblank output lines.
This is one generated English page, also used for calibration, not held-out quality certification. It does not establish general visual reasoning, video accuracy, multilingual OCR, long-context behavior or inference speed. Eager-mode startup, JIT and autotuning messages may affect latency.
After downloading this complete repository to a local directory:
bash
python examples/image_text_smoke.py --model . \ --image calibration/nvfp4-ocr-smoke-page.png --output image-text-runtime.json \ --require-native-fp4 --memory-fraction 0.30Use --memory-fraction 0.065 for the tested Thor recipe. The example requires
compatible CUDA PyTorch, vLLM and Pillow. It uses the retained tokenizer chat
template, does not execute model remote code, and verifies actual worker
kernels before accepting the output. Marlin and emulation are rejected.
axquant_cuda_plan.json, axquant_cuda_manifest.json,
activation_calibration.json, development_runtime_smoke.json, and
SHA256SUMS.txt bind the development artifacts and their scope. The converter
manifest remains runtime_verified=false and quality_certified=false;
separate exact-checkpoint runtime evidence does not become a certificate.
python examples/image_text_smoke.py --model . \ --image calibration/nvfp4-ocr-smoke-page.png --output image-text-runtime.json \ --require-native-fp4 --memory-fraction 0.30