GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ibm-granite
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ibm-granite
Model Tree
ibm-granite/granite-docling-258M
Input Modalities
Output Modalities
Supported Functionality
If you run through 🐥Docling, it will automatically choose the MLX version of the Granite-Docling model. You can select it with the CLI options shown below:
sh
# Convert to HTML and Markdown:docling --to html --to md --pipeline vlm --vlm-model granite_docling "https://arxiv.org/pdf/2501.17887" # accepts files, urls or directories# Convert to HTML including layout visualization:docling --to html_split_page --show-layout --pipeline vlm --vlm-model granite_docling "https://arxiv.org/pdf/2501.17887"
You can also run plain mlx-vlm to generate predictions.
To run with the mlx-vlm CLI, use this command:
sh
pip install mlx_vlmpython -m mlx_vlm.generate --model ibm-granite/granite-docling-258M-mlx --max-tokens 4096 --temperature 0.0 --prompt "Convert this page to docling." --image <path_to_image>
To run with the mlx-vlm python SDK, parse the output as a DoclingDocument and export to various formats (e.g. Markdown, HTML), please refer to the code below.
python
# /// script# requires-python = ">=3.12"# dependencies = [# "docling-core",# "mlx-vlm",# "pillow",# "transformers",# ]# ///import webbrowserfrom pathlib import Pathfrom docling_core.types.doc import ImageRefModefrom docling_core.types.doc.document import DocTagsDocument, DoclingDocumentfrom mlx_vlm import load, stream_generatefrom mlx_vlm.prompt_utils import apply_chat_templatefrom mlx_vlm.utils import load_configfrom transformers.image_utils import load_image# ConfigurationMODEL_PATH = "ibm-granite/granite-docling-258M-mlx"PROMPT = "Convert this page to docling."SHOW_IN_BROWSER = True# Sample images (pick one...)# SAMPLE_IMAGE = "https://huggingface.co/ibm-granite/granite-docling-258M/resolve/main/assets/new_arxiv.png"# SAMPLE_IMAGE = "https://ibm.biz/docling-page-with-list"SAMPLE_IMAGE = "https://ibm.biz/docling-page-with-table"# Load model and processorprint("Loading model...")model, processor = load(MODEL_PATH)config = load_config(MODEL_PATH)# Prepare input image and promptprint("Preparing input...")pil_image = load_image(SAMPLE_IMAGE)formatted_prompt = apply_chat_template(processor, config, PROMPT, num_images=1)# Generate DocTags outputprint("Generating DocTags...\n")output = ""for token in stream_generate(model, processor, formatted_prompt, [pil_image], max_tokens=4096, verbose=False):output += token.textprint(token.text, end="")if "</doctag>" in token.text:breakprint("\n\nProcessing output...")# Create DoclingDocument from generated DocTagsdoctags_doc = DocTagsDocument.from_doctags_and_image_pairs([output], [pil_image])doc = DoclingDocument.load_from_doctags(doctags_doc, document_name="Sample Document")# Export to different formatsprint("\nMarkdown output:\n")print(doc.export_to_markdown())# Save as HTML with embedded imagesoutput_path = Path("./output.html")doc.save_as_html(output_path, image_mode=ImageRefMode.EMBEDDED)print(f"\nHTML saved to: {output_path}")# Open in browserif SHOW_IN_BROWSER:webbrowser.open(f"file:///{str(output_path.resolve())}")