> ## Documentation Index
> Fetch the complete documentation index at: https://www.bolna.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Fireworks AI models in Bolna voice agents

> Connect Fireworks AI serverless models or on-demand GPU deployments to a Bolna voice agent. Full model ID paths, API keys, deployments and autoscaling.

[Fireworks AI](https://fireworks.ai) serves open models through an OpenAI-compatible API, with two tiers:

* **Serverless**: shared, per-token access to popular models. Fastest to start, subject to rate limits.
* **On-demand deployments**: dedicated GPUs for one model, with no rate limits, fast autoscaling and a wider choice of models. You create them with the `firectl` CLI.

The one thing that differs from every other provider here is the **model ID**. Fireworks uses a full resource path, `accounts/fireworks/models/<model>`, and the short name on its own won't work.

***

## Set up Fireworks AI

**1. Create an API key**

In the Fireworks dashboard, open [API keys](https://app.fireworks.ai/settings/users/api-keys), click **Create API key** and store it safely.

**2. Copy the full model ID**

Find a chat model in the Fireworks model library and copy its full ID, for example:

```text theme={"system"}
accounts/fireworks/models/llama-v3p1-8b-instruct
```

Note the Fireworks naming: versions are written with `p` for the decimal point, so Llama 3.1 is `llama-v3p1`.

**3. Choose Fireworks AI in Bolna**

Go to **Settings → Integrations → LLM → Custom LLM → Add model**, or choose **+ Add custom LLM** in an agent's **LLM Provider** dropdown. Select **Fireworks AI**. The hint under the endpoint field reminds you to use the full model ID.

![Bolna Add custom model dialog with Fireworks AI selected, api.fireworks.ai/inference/v1 as the endpoint placeholder and a hint to use accounts/fireworks/models/ model IDs](https://mintcdn.com/bolna-54a2d4fe/0EXypen329Eb5XE2/images/custom-llm/providers/fireworks-selected.png?fit=max\&auto=format\&n=0EXypen329Eb5XE2\&q=85\&s=cda570dc7b5826db64f4d6553494fed1)

Fireworks AI selected, with the full model ID hint

**4. Register the model**

**Dashboard**

* **Model name**: the full path, for example `accounts/fireworks/models/llama-v3p1-8b-instruct`
* **Endpoint URL**: `https://api.fireworks.ai/inference/v1`. The `/inference` segment is required.
* **API key**: your Fireworks key
* **Family**: for example `llama`

![Add custom model dialog with Fireworks AI selected, model accounts/fireworks/models/llama-v3p1-8b-instruct, endpoint api.fireworks.ai/inference/v1, masked key and family llama](https://mintcdn.com/bolna-54a2d4fe/aV72vktown5Rn3f1/images/custom-llm/providers/fireworks-filled.png?fit=max\&auto=format\&n=aV72vktown5Rn3f1\&q=85\&s=2bdb2335018a1c9b8b9a7c2f39a7f801)

Fireworks AI serverless model, filled in

**API**

```bash theme={"system"}
curl --request POST \
  --url https://api.bolna.ai/user/model/custom \
  --header 'Authorization: Bearer <BOLNA_API_KEY>' \
  --header 'Content-Type: application/json' \
  --data '{
    "custom_model_name": "accounts/fireworks/models/llama-v3p1-8b-instruct",
    "custom_model_url": "https://api.fireworks.ai/inference/v1",
    "custom_model_key": "<FIREWORKS_API_KEY>",
    "custom_model_family": "llama"
  }'
```

**5. Select it on the agent**

Choose **custom** on the **Intelligence** tab, pick the model and save.

***

## Moving to an on-demand deployment

When a campaign starts hitting serverless rate limits, or you need a model that isn't on serverless, create an on-demand deployment:

```bash theme={"system"}
firectl create deployment accounts/fireworks/models/gpt-oss-120b \
  --deployment-shape default \
  --min-replica-count 0 \
  --scale-to-zero-window 5m
```

`firectl` prints the deployment's `Name`, in the form `accounts/<account-id>/deployments/<deployment-id>`. Fireworks routes a request to your deployment when the model ID ends in `#` plus that name:

```text theme={"system"}
accounts/fireworks/models/gpt-oss-120b#accounts/<account-id>/deployments/<deployment-id>
```

Register that string as the model name in Bolna, with the same endpoint URL and API key.

> [!WARNING]
>
> `--min-replica-count 0` lets the deployment scale to zero, and the next call then waits for a GPU to start. For live voice traffic, keep at least one replica with `--min-replica-count 1`.

***

## Troubleshooting

> **Model not found**
>
> You probably entered a short name such as `llama-v3p1-8b-instruct`. Use the full `accounts/fireworks/models/...` path.

> **404 on chat completions**
>
> The endpoint must include `/inference`: `https://api.fireworks.ai/inference/v1`.

> **429 errors on serverless**
>
> Serverless has rate limits. Move the agent to an on-demand deployment, which has none.

***

## Related

* [Custom LLM inference](https://www.bolna.ai/docs/custom-llm-inference)
* [Together AI](https://www.bolna.ai/docs/custom-llm-inference/together.md): a large serverless catalog of open models
* [Baseten](https://www.bolna.ai/docs/custom-llm-inference/baseten.md): Model APIs and dedicated deployments


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.