Skip to content

Fallbacks

Stay up when a provider goes down

List your models in the order you want them tried. If the first one can't answer, the next one does.

Start buildingRead the docs
Language
curl https://api.hopscotchlabs.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_HOPSCOTCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "profile/customer-support",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

How fallbacks behave

  1. Backups in your order

    Hopscotch tries the models you list in the order you set.

  2. Change models without a deploy

    Your code sends the profile name. You change its models in the dashboard.

  3. See which model answered

    Activity shows the profile and provider for every request.

How it works

  1. List your models

    Pick a primary model and its backups in the dashboard.

  2. Use it in your code

    Set the profile you want in your code. Edit its models in the dashboard.

  3. Hopscotch tries them in order

    If a model can't serve the request, the next one does. A malformed request goes straight back to you.

Where it helps

  1. A provider outage at peak hours

    A backup from another maker keeps your app answering.

  2. Model IDs hardcoded across teams

    Teams send a profile name. You change the models in the dashboard, and nobody redeploys.

  3. Trying a new model safely

    Put it behind your primary as a backup. Activity shows each request it answers.

Fallback questions

How many models and profiles can I have?

Up to five models per profile: one primary and up to four backups. A workspace can have up to 20 profiles.

Which errors move a request to the next model?

Errors that mean the model couldn't serve it, such as a provider outage. A request with a problem of its own, like a malformed body, comes back to you.

What if every model in the profile is down?

The request is refused with HTTP 503 and the code no_provider_configured. A refused request costs nothing.

Does a fallback cost more?

It can. You pay the list price of the model that answered, so a pricier backup costs more per request.

Related

Put a backup behind your main model

Build a routing profile and send its name as the model.

Start buildingDocs