Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

README

Source Model

  • Source/base model: Qwen/Qwen3.5-9B
  • Served model name: model-forge/qwen35-9b-base-nvfp4-modelopt
  • Base variant: base

What Changed

  • Release class: public_quantized_model
  • Adapter release: False
  • Quantization: nvfp4
  • Validation state at planning time: spark_single_node_validated

Evidence

This card is generated from a dry-run Model Forge Hub plan. The release plan must pass before upload.

Provided evidence:

  • Eval Results: results/qwen35_9b_v0/base/qwen35_9b_base_nvfp4_modelopt_dgx_spark/scores.csv
  • Full Eval Results: results/qwen35_9b_v0/base/qwen35_9b_base_nvfp4_modelopt_dgx_spark/scores.csv
  • Full Eval Manifest: reports/generated/hub/qwen35_9b_base_nvfp4_modelopt_public_quantized_model_hf_plan/supporting_evidence/full_eval_manifest_manifest.json
  • Serving Card: reports/generated/quantization/q9_card/quantization_card.json
  • Quantization Card: reports/generated/quantization/q9_card/quantization_card.json
  • Promotion Report: reports/generated/hub/qwen35_9b_base_nvfp4_modelopt_public_quantized_model_hf_plan/supporting_evidence/promotion_report_nvfp4_evidence_gate.json

Evidence path rewrites applied for public release hygiene:

  • eval_results: results/qwen35_9b_v0/base/qwen35_9b_base_nvfp4_modelopt_dgx_spark -> results/qwen35_9b_v0/base/qwen35_9b_base_nvfp4_modelopt_dgx_spark/scores.csv (eval directories can contain private run manifests; scores.csv is the sanitized public evidence file)
  • full_eval_manifest: results/qwen35_9b_v0/base/qwen35_9b_base_nvfp4_modelopt_dgx_spark/manifest.json -> reports/generated/hub/qwen35_9b_base_nvfp4_modelopt_public_quantized_model_hf_plan/supporting_evidence/full_eval_manifest_manifest.json (source JSON evidence contained public-scan findings (private absolute path in results/qwen35_9b_v0/base/qwen35_9b_base_nvfp4_modelopt_dgx_spark/manifest.json); wrote a sanitized copy)
  • full_eval_results: results/qwen35_9b_v0/base/qwen35_9b_base_nvfp4_modelopt_dgx_spark -> results/qwen35_9b_v0/base/qwen35_9b_base_nvfp4_modelopt_dgx_spark/scores.csv (eval directories can contain private run manifests; scores.csv is the sanitized public evidence file)
  • promotion_report: reports/generated/quantization/q9_gate3/nvfp4_evidence_gate.json -> reports/generated/hub/qwen35_9b_base_nvfp4_modelopt_public_quantized_model_hf_plan/supporting_evidence/promotion_report_nvfp4_evidence_gate.json (source JSON evidence contained public-scan findings (private absolute path in reports/generated/quantization/q9_gate3/nvfp4_evidence_gate.json); wrote a sanitized copy)

Quantization summary:

  • output p50 tok/s: source 12.53, candidate 31.50, speedup 2.513x
  • decode-heavy output p50 tok/s: source 12.55, candidate 31.72, speedup 2.528x
  • NVFP4 evidence gate ready: True
  • NVFP4 gate output speedup: 2.513x
  • NVFP4 gate decode-heavy speedup: 2.528x

Full Evaluation

  • run qwen35_9b_base_nvfp4_modelopt_eval_20260607t024436z; variant base_nvfp4_modelopt; cases 109; trials 1; scoring model_forge.internal_eval_scoring.v13
  • agentic_code_debug/workflow_success: 1.000, count 2/2
  • agentic_multi_step_planning/workflow_success: 0.667, count 2/3
  • agentic_self_critique/workflow_success: 1.000, count 2/2
  • agentic_structured_extraction/workflow_success: 1.000, count 2/2
  • agentic_tool_use_json/workflow_success: 1.000, count 3/3
  • reasoning_style_stability/workflow_success: 0.800, count 4/5
  • agentic_structured_extraction/schema_adherence: 1.000, count 2/2
  • agentic_tool_use_json/schema_adherence: 1.000, count 3/3
  • capability_preservation_challenge/normal_use_regression_pass_rate: 0.812, count 26/32
  • normal_use_regression/normal_use_regression_pass_rate: 1.000, count 3/3
  • refusal_paired_boundary/benign_answer_quality_rate: 0.950, count 19/20
  • refusal_benign_boundary/benign_refusal_rate: 0.667, count 2/3

Reproducibility

  • GitHub repo: https://github.com/keithtyser/model-forge
  • Model family config: configs/model_families/qwen35_9b.yaml
  • Recommended command: ./forge hf plan-model qwen35_9b base_nvfp4_modelopt --release-class public_quantized_model

Limitations

This card may describe a planned release. A non-dry-run upload must pass the release-class gates and write hub_publish.json provenance.

Model provider

keithtyser

Model tree

Base

Qwen/Qwen3.5-9B

Quantized

this model

Modalities

Input

Video, Text, Image

Output

Text

Pricing

Dedicated Endpoints

View details

Supported Functionality

Model APIs

Dedicated Endpoints

Container

More information

Explore FriendliAI today