GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
andrewluo
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
andrewluo
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
Use one image and this instruction in a single user message, with no extra system instruction:
text
Describe this document figure accurately and usefully. Include clearly legible labels, quantities, units, and important visual relationships. State uncertainty when details cannot be read. Do not invent context or values.Use temperature 0, seed 0, up to 2,048 output tokens, and thinking disabled. The expected response is a plain-text caption, not JSON. The serving configs cap the image processor at 2,097,152 pixels and the full context at 8,192 tokens. A complete OpenAI-compatible request is in serving/caption-request.json; replace its image placeholder with real image data.
The serving/baseline/config.yaml file targets one H100 with BF16 weights using the pinned Linux amd64 vLLM 0.29.0 container. The target checkpoint revision is pinned after upload verification. The private weights are mirrored to /models/target and served as figure-caption-32826 through /v1/chat/completions.
HF_TOKEN containing a Hugging Face read token with access to this private repository. The weights configuration explicitly references that secret.bash
hf download andrewluo/Qwen3.5-4B-figure-caption-32826-merged --include 'serving/**' --local-dir figure-caption-servingbaseten model push --dir figure-caption-serving/serving/baselineThe download uses your logged-in Hugging Face account or HF_TOKEN. The push starts a paid GPU deployment. This repository publication did not deploy a GPU endpoint.
Baseten exposes the deployed model through its OpenAI-compatible sync URL. Use the deployment's URL, your Baseten API key, model name figure-caption-32826, and the included request body. See Baseten's vLLM guide for authentication and invocation.
serving/dflash-7/config.yaml and serving/dflash-15/config.yaml enable DFlash with a pinned z-lab/Qwen3.5-4B-DFlash draft and standard target verification. Use either directory instead of baseline when deploying. The baseline disables speculative decoding. Switching configurations requires redeployment.
The draft's hidden size, vocabulary, layer count, feature-layer indices, and mask token are compatible with the target architecture. Target embeddings are unchanged. The draft was trained against the original base model rather than this continuation. Actual acceptance rate, caption latency, and throughput need measurement on the intended H100 workload; these settings are candidates, not proven optima or guaranteed speedups. No quantization or lossy acceptance is enabled. The original auxiliary MTP tensors are preserved but are not used or evaluated by these DFlash configurations.
The export uses float32 W + 2 * (B.T @ A.T) and rounds once to BF16. All 128 adapted projection matrices are checked against separate base-plus-adapter computations and independent sampled delta calculations. All 738 serialized tensors round-trip exactly; the other 610 retain their original bytes and layouts. Uploaded file sizes and SHA-256 values are verified against the exported files.
A full standard Transformers Qwen3_5ForConditionalGeneration.from_pretrained load and image-plus-text caption generation are checked on CPU. The Baseten configuration schema is validated locally. H100/CUDA execution and DFlash speed have not been runtime-tested. The final validation loss of 1.004185 covers 32 sequences from the unchanged validation pool and is not a direct measure of production caption accuracy.
See conversion_validation.json, runtime_validation.json, baseten_validation.json, dflash_compatibility.json, training_metadata.json, and manifest.json. No training images, production evaluation images, private evaluation captions, or optimizer state are included.
Architecture support: vLLM supported models. DFlash implementation: vLLM 0.29.0.
A uniform random sample of 250 figures from the saved 1,000-figure production cohort was evaluated with blinded gpt-6-luna judgments. On the same 238 figures with stored production captions, mean overall scores were 7.10/10 for production, 7.96 for the previous chart checkpoint, and 8.41 for this checkpoint. This checkpoint scored higher than the previous checkpoint on 102 cases, equally on 87, and lower on 49.
A 25-case reversed-order audit agreed on the winner set in 17 cases. The cohort was used in prior development evaluation, so this is not a fresh holdout or human-verified accuracy result. Eleven duplicate primary decisions from an early streaming issue were excluded using the earliest timestamp for each figure; every selected figure contributes once. Only sanitized aggregate results are published in evaluation_summary.json.
hf download andrewluo/Qwen3.5-4B-figure-caption-32826-merged --include 'serving/**' --local-dir figure-caption-servingbaseten model push --dir figure-caption-serving/serving/baseline