FriendliAI Meets Vercel AI Gateway

Vercel AI Gateway provides a single API for accessing AI models across multiple inference providers, including FriendliAI. This lets you run open-weight models supported on FriendliAI through your existing Gateway integration. You can prefer FriendliAI while retaining other providers as fallbacks, or restrict requests to FriendliAI to evaluate its performance on your workload.

This guide uses GLM-5.3 to show how to send requests to FriendliAI, control provider fallback, and choose between Gateway credits and your FriendliAI account for inference billing. It also covers how to verify which provider handled the request and whether Gateway-managed credentials or your own FriendliAI API key were used.

Why We Brought FriendliAI to Vercel AI Gateway

Providers serving the same model can differ in latency, throughput, reliability, and cost. We worked with Vercel to bring FriendliAI to AI Gateway so developers can evaluate its inference performance on their own workloads and choose a provider that meets their application’s requirements.

Latency illustrates why provider choice matters. On Vercel AI Gateway, FriendliAI recorded an average hourly median time to first token (TTFT) of 266 ms for GLM-5.3, the lowest among 22 providers serving the model.

Over the past week, FriendliAI was 19.8x faster than the average of other providers, and 211.8x faster than the slowest provider. Explore Figure 1 to see how latency gaps and provider rankings changed over time.

Send Your First Request

This example uses GLM-5.3. For the current list of supported models, see FriendliAI’s model catalog on Vercel AI Gateway.

You’ll need:

• Node.js 22 or later.
• An AI Gateway API key.

Install the latest version of the ai package:

npm install ai@latest

Then save your Gateway API key in .env.local:

AI_GATEWAY_API_KEY=your_gateway_api_key

Save the following as friendli-gateway.mjs. It requests GLM-5.3 and uses order: ['friendli'] to prefer FriendliAI while allowing other eligible providers as fallbacks.

import { gateway, generateText } from 'ai';

const res = await generateText({
  model: gateway('zai/glm-5.3'),
  prompt: 'Explain Vercel AI Gateway in two sentences.',
  maxOutputTokens: 4096,
  providerOptions: {
    gateway: {
      order: ['friendli'],
    },
  },
});

console.log(res.text);
console.log(
  JSON.stringify(res.providerMetadata?.gateway?.routing, null, 2),
);

Run it with the environment file loaded explicitly:

node --env-file=.env.local friendli-gateway.mjs

A successful run prints the generated text followed by routing metadata.

Control Provider Fallback

Use providerOptions.gateway to control whether Gateway can fall back to another provider if FriendliAI is unavailable or unable to complete a request.

To evaluate FriendliAI's performance without falling back to other providers, replace order with only in the providerOptions configuration:

providerOptions: {
  gateway: {
    only: ['friendli'],
  },
},

To configure fallback behavior across models, see model fallbacks.

Bring Your Own FriendliAI Key (BYOK)

You can use FriendliAI through Vercel AI Gateway with either Gateway credits or your own FriendliAI API key (BYOK). To keep inference billing through Vercel without creating a FriendliAI account, use Gateway credits. To use your own FriendliAI account and credits, bring your own FriendliAI API key (BYOK). Gateway authenticates requests with your key without charging an additional fee.

Set up BYOK

1. Create a FriendliAI API key in Friendli Suite → Settings → API Keys.
2. Open AI Gateway → Bring Your Own Key (BYOK) in your Vercel team’s dashboard.
3. Find FriendliAI, click Add, and enter your key. Keep Enabled turned on.
4. Click Test Key to validate and add the credentials.

Gateway automatically uses the saved key for FriendliAI requests across your Vercel team. You can keep using AI_GATEWAY_API_KEY to authenticate your application to Gateway without changing your code.

For requests that need a different provider key, see request-scoped BYOK.

If a request using your FriendliAI key fails, Gateway retries with system credentials, and that fallback usage is billed against your Gateway credits. This can happen even with only: ['friendli'].

Verify the Provider and Credentials

To identify which provider handled the request, look for success: true under modelAttempts[].providerAttempts[] in the routing metadata printed by the example.

{
  "provider": "friendli",
  "credentialType": "system",
  "success": true
}

In the response above, friendli identifies the provider, and system means the Vercel AI Gateway used its own credentials and billed inference to Gateway credits. If you see byok in the credentialType, it means your FriendliAI key was used, and inference was billed to your FriendliAI account. You can also inspect the provider and credential source in Gateway Logs.

Try FriendliAI with Your Workload

Once you've verified your provider and credentials, test GLM-5.3 with your application or coding agent. Measure latency, throughput, and cost at the context lengths and concurrency you expect in production.

Try FriendliAI on Vercel AI Gateway →