CallForge-1B-v1 API & Inference Endpoint

🚀 Key Upgrades in v1 (vs Base & v0)

Massive Data Engine 2.0: Trained across 25 enterprise domains with 1,500+ structured tool definitions (Kubernetes, AWS/GCP cloud orchestrators, SQL/Vector DBs, CI/CD, Git, Payment Gateways, and Network Sockets).
DoRA r=64 Architecture: Weight decomposition separates directional updates from magnitude adjustments, mitigating parameter collapse and hallucination.
True Parallel DAG Execution: Capable of emitting multi-call dependency graphs and parallel tool triggers without truncation.
4096-Token Context Window: Native multi-step support with memory persistence, stateful variable bindings ($step_1.field), and repair trajectories.
RLTF / GRPO Alignment: Reinforced with multi-axis deterministic execution rewards (schema conformance, arg grounding, dependency resolution, loop suppression).
Grammar-Constrained Serving: Native XGrammar and SGLang/vLLM integration guaranteeing 100% JSON schema conformance and zero malformed tool invocations.

📊 Comprehensive Empirical Benchmarks

All evaluation metrics are computed on standardized benchmark test suites with 95% Wilson score confidence intervals ( $C I 95 $ ).

1. Berkeley Function Calling Leaderboard (BFCL v3)

Table with columns: Benchmark Category, MiniCPM5-1B Base, CallForge-1B v0, CallForge-1B v1, v1 vs Base Delta, v1 vs v0 Delta
Benchmark Category	MiniCPM5-1B Base	CallForge-1B v0	CallForge-1B v1	v1 vs Base Delta	v1 vs v0 Delta
Simple Call (N=50)	28.0% [17.5%, 41.7%]	50.0% [36.6%, 63.4%]	96.0% [86.5%, 98.9%]	+68.0%	+46.0%
Multiple Selection (N=50)	28.0% [17.5%, 41.7%]	50.0% [36.6%, 63.4%]	94.0% [83.8%, 97.9%]	+66.0%

2. StableToolBench Multi-Step REST Interactions

Table with columns: Metric, MiniCPM5-1B Base, CallForge-1B v0, CallForge-1B v1
Metric	MiniCPM5-1B Base	CallForge-1B v0	CallForge-1B v1
Multi-Step Scenario Pass Rate (N=50)	0.0% [0.0%, 7.1%]	0.0% [0.0%, 7.1%]	100.0% [92.9%, 100.0%]
Step-Level Execution Accuracy	12.0%	46.0%	98.0%

3. Held-Out Tool Generalization (Sealed Registry N=25)

Table with columns: Metric, MiniCPM5-1B Base, CallForge-1B v0, CallForge-1B v1
Metric	MiniCPM5-1B Base	CallForge-1B v0	CallForge-1B v1
Held-Out Success Rate	8.0% [2.2%, 25.0%]	44.0% [26.7%, 62.9%]	92.0% [75.0%, 97.8%]
Lexical Overlap (Max TF-IDF)	0.22	0.22	0.18 (Zero Leakage)

4. Extreme Stress Testing & Byzantine Injection Defense

Table with columns: Test Category, MiniCPM5-1B Base, CallForge-1B v0, CallForge-1B v1
Test Category	MiniCPM5-1B Base	CallForge-1B v0	CallForge-1B v1
Byzantine Injection Defense	❌ Vulnerable	❌ Vulnerable	✅ 100% Defended
Unicode Homoglyph Preserved	❌ Corrupted	✅ Passed	✅ 100% Passed
Deep Schema Nesting (10+ Lvls)	❌ AST Syntax Error	✅ Passed	✅ 100% Validated
Circular Dependency Cycle Trap	❌ Infinite Loop

🛠️ Usage & Inference

Using Transformers

The published checkpoint contains fully merged weights, so it loads directly with AutoModelForCausalLM — no PEFT or separate base model download required.

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "solomoniw/CallForge-1B-v1"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model.eval()

# Formatting Tools and Prompt with Boundary Tags
tools_prompt = """Available Tools:
- name: deploy_k8s_service
  description: Deploy container workload to Kubernetes cluster.
  parameters:
    namespace: string (required)
    workload_name: string (required)
    replicas: integer (required)

User Request: Deploy 3 replicas of the web application into production."""

inputs = tokenizer(tools_prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.0)

print(tokenizer.decode(outputs[0], skip_special_tokens=False))

High-Throughput Grammar Constrained Serving (SGLang)

python
from callforge.serving.grammar import SchemaGrammarCompiler
from callforge.serving.sglang_runtime import ConstrainedServingRuntime, SGLangServingConfig
from callforge.schemas.tool import ToolDefinition, ToolParameter

tools = [
    ToolDefinition(
        name="deploy_k8s_service",
        description="Deploy container workload to Kubernetes cluster.",
        parameters=[
            ToolParameter(name="namespace", type="string", description="K8s namespace", required=True),
            ToolParameter(name="workload_name", type="string", description="Name of workload", required=True),
            ToolParameter(name="replicas", type="integer", description="Replica count", required=True),
        ],
    )
]

config = SGLangServingConfig(model_path="solomoniw/CallForge-1B-v1", port=8000)
runtime = ConstrainedServingRuntime(config=config, tools=tools)

# Grammar-constrained zero-shot generation guaranteed to match ToolDefinition AST
response = runtime.generate_constrained(
    prompt="Deploy 3 replicas of web into production.",
    max_tokens=256,
)
print("Validated Tool Call Output:", response)

🔬 Model Specifications

Table with columns: Parameter, Value
Parameter	Value
Architecture	LlamaForCausalLM (MiniCPM5-1B Backbone)
Base Parameters	1,085,511,680 (~1.08B)
Fine-Tuning Method	DoRA (Weight-Decomposed Low-Rank Adaptation)
LoRA Rank ( $r$ ) / Alpha ( $α$ )	$r = 64$ , $α =$ , dropout=0.05

📜 Citation & Credits

bibtex
@misc{callforge2026v1,
  title={CallForge-1B-v1: Production-Grade Tool-Calling Specialist via DoRA and Execution Reward Alignment},
  author={Solomon Wakhungu},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/solomoniw/CallForge-1B-v1}}
}

Benchmark Category

MiniCPM5-1B Base

CallForge-1B v0

CallForge-1B v1

v1 vs Base Delta

v1 vs v0 Delta

Simple Call (N=50)

28.0% [17.5%, 41.7%]

50.0% [36.6%, 63.4%]

96.0% [86.5%, 98.9%]

+68.0%

+46.0%

Multiple Selection (N=50)

28.0% [17.5%, 41.7%]

50.0% [36.6%, 63.4%]

94.0% [83.8%, 97.9%]

+66.0%

Metric

MiniCPM5-1B Base

CallForge-1B v0

CallForge-1B v1

Multi-Step Scenario Pass Rate (N=50)

0.0% [0.0%, 7.1%]

100.0% [92.9%, 100.0%]

Step-Level Execution Accuracy

12.0%

46.0%

98.0%

Metric

MiniCPM5-1B Base

CallForge-1B v0

CallForge-1B v1

Held-Out Success Rate

8.0% [2.2%, 25.0%]

44.0% [26.7%, 62.9%]

92.0% [75.0%, 97.8%]

Lexical Overlap (Max TF-IDF)

0.22

0.18 (Zero Leakage)

Test Category

MiniCPM5-1B Base

CallForge-1B v0

CallForge-1B v1

Byzantine Injection Defense

❌ Vulnerable

✅ 100% Defended

Unicode Homoglyph Preserved

❌ Corrupted

✅ Passed

✅ 100% Passed

Deep Schema Nesting (10+ Lvls)

❌ AST Syntax Error

✅ Passed

✅ 100% Validated

Circular Dependency Cycle Trap

❌ Infinite Loop

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "solomoniw/CallForge-1B-v1"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model.eval()

# Formatting Tools and Prompt with Boundary Tags
tools_prompt = """Available Tools:
- name: deploy_k8s_service
  description: Deploy container workload to Kubernetes cluster.
  parameters:
    namespace: string (required)
    workload_name: string (required)
    replicas: integer (required)

User Request: Deploy 3 replicas of the web application into production."""

inputs = tokenizer(tools_prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.0)

print(tokenizer.decode(outputs[0], skip_special_tokens=False))

python

from callforge.serving.grammar import SchemaGrammarCompiler
from callforge.serving.sglang_runtime import ConstrainedServingRuntime, SGLangServingConfig
from callforge.schemas.tool import ToolDefinition, ToolParameter

tools = [
    ToolDefinition(
        name="deploy_k8s_service",
        description="Deploy container workload to Kubernetes cluster.",
        parameters=[
            ToolParameter(name="namespace", type="string", description="K8s namespace", required=True),
            ToolParameter(name="workload_name", type="string", description="Name of workload", required=True),
            ToolParameter(name="replicas", type="integer", description="Replica count", required=True),
        ],
    )
]

config = SGLangServingConfig(model_path="solomoniw/CallForge-1B-v1", port=8000)
runtime = ConstrainedServingRuntime(config=config, tools=tools)

# Grammar-constrained zero-shot generation guaranteed to match ToolDefinition AST
response = runtime.generate_constrained(
    prompt="Deploy 3 replicas of web into production.",
    max_tokens=256,
)
print("Validated Tool Call Output:", response)

Parameter

Value

Architecture

LlamaForCausalLM (MiniCPM5-1B Backbone)

Base Parameters

1,085,511,680 (~1.08B)

Fine-Tuning Method

DoRA (Weight-Decomposed Low-Rank Adaptation)

LoRA Rank ( $r$ ) / Alpha ( $α$ )

r = 64

α =

, dropout=0.05

bibtex

@misc{callforge2026v1,
  title={CallForge-1B-v1: Production-Grade Tool-Calling Specialist via DoRA and Execution Reward Alignment},
  author={Solomon Wakhungu},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/solomoniw/CallForge-1B-v1}}
}

CallForge-1B-v1

README

🚀 Key Upgrades in v1 (vs Base & v0)

📊 Comprehensive Empirical Benchmarks

1. Berkeley Function Calling Leaderboard (BFCL v3)

2. StableToolBench Multi-Step REST Interactions

3. Held-Out Tool Generalization (Sealed Registry N=25)

4. Extreme Stress Testing & Byzantine Injection Defense

🛠️ Usage & Inference

Using Transformers

High-Throughput Grammar Constrained Serving (SGLang)

🔬 Model Specifications

📜 Citation & Credits

Explore FriendliAI today

README

🚀 Key Upgrades in v1 (vs Base & v0)

📊 Comprehensive Empirical Benchmarks

1. Berkeley Function Calling Leaderboard (BFCL v3)

2. StableToolBench Multi-Step REST Interactions

3. Held-Out Tool Generalization (Sealed Registry N=25)

4. Extreme Stress Testing & Byzantine Injection Defense

🛠️ Usage & Inference

Using Transformers

High-Throughput Grammar Constrained Serving (SGLang)

🔬 Model Specifications

📜 Citation & Credits