No quantization-container correction was needed.
This family has no n-gram tensors; no n-gram file or declaration was added.
This CUDA pack is outside MLX/oMLX/MTPLX export scope.
See runtime_audit.json for pinned config/index/header
bindings, architecture, physical-format findings, and applied corrections.
This is development evidence; no quality, MTP exactness, speed, or certification
claim is added. Historical evidence stays bound to its original revision.
AX-Unlimited-OCR-3B-MoE-CUDA-AXQ-NVFP4-W4A16
Development preview. This is the trained Unlimited-OCR checkpoint
converted with AXQuant-owned round-to-nearest (RTN) NVFP4 weight packing.
No AWQ algorithm or AWQ checkpoint is used.
Selected language attention, MLP and individual expert projections use
NVFP4 W4A16: E2M1 FP4 weights, E4M3FN block scales (16 values per block),
and BF16 activations. Vision, routers, embeddings, normalization and LM head
retain their original BF16 tensor payloads. Activation quantization is disabled.
Source and payload
- Original source: baidu/Unlimited-OCR,
immutable revision
07dea832e22aefee32ad281d4b80551282e1c168 (MIT).
- Original weight SHA-256 was verified against the pinned upstream Hub object.
- Original parameter count: 3,336,106,240.
- NVFP4 parameter count: 2,602,844,160, across 2,196 tensors.
- Protected source-precision tensors: 514, verified for dtype and value equality.
- Weight file: 2,931,515,952 bytes (about 2.93 GB / 2.73 GiB).
- Nominal selected-weight payload: 4.5 bits/value plus one FP32 scale per matrix.
Protected BF16 tensors increase the overall checkpoint bits per parameter.
The factory export used AXQuant's explicit numpy-reference encoder.
AXQuant's CUDA encoder is separately covered by real-GPU byte parity tests.
The checkpoint targets NVIDIA inference; the manifest identifies the actual
conversion backend. No MLX-to-NVIDIA reinterpretation is performed.
Parallel Q/K/V and gate/up projections share global scales for runtime fusion.
Protected vision/audio namespaces also have conservative runtime ignore rules,
covering vision wrappers that rename transformer paths to encoder.
Development runtime evidence
The same weight file, config and self-generated English test page were checked
with public vLLM 0.25.1, PyTorch 2.11.0+cu130 and CUDA 13.0.
The kernels were MarlinNvFp4LinearKernel and the MARLIN NVFP4 MoE backend.
Table with columns: GPU, NVFP4 checkpoint load, Expected text lines with min_tokens=16| GPU | NVFP4 checkpoint load | Expected text lines with min_tokens=16 |
|---|
| GeForce RTX 5090 | Passed | All three present |
| Thor | Passed | All three present |
The RTX 5090 BF16 control also recognized all three lines with that token minimum.
The three expected lines were AXQuant NVFP4, Invoice 12345, and
Total USD 42.50. See development_runtime_smoke.json for exact output,
input image digest and request settings. These are single-page development
checks; trained-model/OCR quality and performance are not certified.
Known early-EOS behavior: the RTX 5090 run with the default minimum of
zero tokens ended before producing the page contents. BF16 control runs also
ended early. For this nonempty smoke page, min_tokens=16 produced the full
expected text. The reproduction command below records that setting. A token
minimum is a request setting, not a general OCR accuracy guarantee.
Reproduce the included smoke
The corrected example enables chunked prefill and removes only the four
image build metadata variables. The unchanged checkpoint was retested with
chunked prefill on RTX 5090 and Thor; all three expected page lines were
present. Exact outputs and image/configuration digests are in
development_runtime_smoke.json.
hf download AutomatosX/AX-Unlimited-OCR-3B-MoE-CUDA-AXQ-NVFP4-W4A16 --local-dir model-nvfp4python model-nvfp4/examples/ocr_smoke.py \ --model model-nvfp4 --image model-nvfp4/examples/ocr-smoke-page.png \ --output ocr-smoke.json --memory-fraction 0.30
Use --memory-fraction 0.035 for the tested Thor recipe. This development
example uses minimum 16 tokens, deterministic sampling, native vLLM OCR
processors, Torch SDPA vision attention and Triton text attention. It rejects
repeated nonblank output lines. This small page does not establish general
OCR quality or correct layout markup. Actual documents and concurrency need
independent validation and memory sizing.
For verified native FP4 computation, use the calibrated
W4A4 successor. This W4A16 checkpoint
uses Marlin weight-only FP4 compression. Thor has native FP4 hardware; the
Marlin warning describes this execution path. The W4A4 successor verifies
native CUTLASS Linear and native quantized MoE kernel coverage on both GPUs.
No AWQ is used in either format. This guidance update does not change weights,
quantization configuration, original provenance or historical certificates.
Artifact and claim scope
axquant_cuda_plan.json: exact source content binding, protected allocation,
shared-scale groups and unmeasured RTN label.
axquant_cuda_manifest.json: core output file digests and development status.
provenance.json: immutable source identity and converter details.
SHA256SUMS.txt: complete publication payload digests.
examples/: the identical self-generated test page and reproduction script.
Manifest fields remain runtime_verified=false and quality_certified=false.
The separate smoke evidence describes the exact cases actually exercised;
it does not promote this preview to toolkit-wide runtime qualification,
Tier 1/2 certification, W4A4 activation quantization or a GA release.