​
1. Log In or Sign Up

  • If you have an account, log in using your preferred SSO or email/password combination.
  • If you’re new to FriendliAI, create an account for free.

​
2. Access Friendli Dedicated Endpoints

  • On your left sidebar, find the “Dedicated Endpoints” option.
  • Click the option to access the endpoint list page.

​
3. Prepare Your Model

  • Choose a model that you wish to serve from HuggingFace, Weights & Biases, or upload your custom model on our cloud.

​
4. Deploy Your Endpoint

  • Deploy your endpoint, using the model of your choice prepared from step 3, and the instance equipped with your desired GPU specification.
  • You can also configure your replicas and the max-batch-size for your endpoint.

​
5. Generate Responses

  • You can generate your responses in two ways: playground and endpoint URL.
  • Try out and test generating responses on your custom model using a chatGPT-like interface at the playground tab.
  • For general usages, send queries to your model through our API at the given endpoint address, accessible on the endpoint information tab.

​
Generating Responses Through the Endpoint URL

Refer to this guide for general instructions on Friendli Token.

# Send inference request to a running Friendli Dedicated Endpoints using a `curl` command.

curl -X POST https://api.friendli.ai/dedicated/v1/completions \
  -H "Content-Type: application/json" \
  -H "X-Friendli-Team: $TEAM_ID" \
  -H "Authorization: Bearer $FRIENDLI_TOKEN" \
  -d '{
    "model": "$ENDPOINT_ID",
    "prompt": "Python is a popular",
    "min_tokens": 20,
    "max_tokens": 30,
    "top_k": 32,
    "top_p": 0.8,
    "n": 3,
    "no_repeat_ngram": 3,
    "ngram_repetition_penalty": 1.75
  }'

For a more detailed tutorial for your usage, please refer to our tutorial for using HuggingFace models and W&B models.