> ## Documentation Index
> Fetch the complete documentation index at: https://www.bolna.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B test voice agent prompts and versions

> A/B test a Bolna voice agent: run a prompt, voice or model change as an agent version on some calls, a batch or a workflow agent node, compare results, make the winner live.

To A/B test a Bolna voice agent, save each variant as an [agent version](https://www.bolna.ai/docs/agent-setup/agent-versions.md), send part of your calls to each version, then compare the results by `agent_version_id` and make the winner live. Your live agent keeps answering every other call while the test runs.

A version is a full copy of the agent's setup, so you can test any change: a new prompt or welcome message, a different voice, LLM or transcriber, or other call settings. Each call runs on exactly one version, and records it as the call's `agent_version_id`.

## How to A/B test a voice agent

1. **Save the variants.** Edit the agent and [save each change as a new version](https://www.bolna.ai/docs/agent-setup/agent-versions.md#save-as-a-new-version). The live version is usually your control.
2. **Split the calls.** Pass `agent_version_id` on each [`POST /call`](#split-individual-calls), [split a batch](#split-a-batch) across 2 to 4 versions by percentage, or [split a workflow agent node](#split-a-workflow-agent-node) across its versions by weight.
3. **Compare the results.** Filter call history and executions by version, and compare answer rate, duration, cost and extracted data. See [Compare the results](#compare-the-results).
4. **Make the winner live.** Every call that doesn't ask for a version then uses it. See [Pick a winner](#pick-a-winner).

There are three ways to split calls across versions:

| | [Individual calls](#split-individual-calls) | [Batches](#split-a-batch) | [Workflow agent nodes](#split-a-workflow-agent-node) |
| - | - | - | - |
| **How** | Pass `agent_version_id` on each `POST /call` | Set the split when you upload the batch, in the dashboard or with `version_split` on `POST /batches` | Turn on the split on the agent node, in the workflow editor or with `version_split` in the node's config |
| **Who decides each call's version** | You, in your own code | Bolna, by the percentages you set | Bolna, by the weights on the node |
| **Versions** | Any number | 2 to 4 | 2 or more |
| **Best for** | Calls you trigger from your own system | Outbound campaigns from a CSV | Multi-step journeys |

Calls that don't ask for a version, including all inbound calls, keep using the live version.

> [!NOTE]
>
> Save each variant as its own version first, see [Save as a new version](https://www.bolna.ai/docs/agent-setup/agent-versions.md#save-as-a-new-version). You'll need each version's `id`, from the version switcher or from [List agent versions](https://www.bolna.ai/docs/api-reference/agent/v2/versions/list.md).

***

## Split individual calls

[Make call](https://www.bolna.ai/docs/api-reference/calls/make.md#call-a-specific-agent-version) takes an optional `agent_version_id`. The call runs on that version's prompts, voice, models and settings, and calls without it run on the live version. To A/B test, choose a version for each call in your own code and pass it.

Assign each contact a version once and reuse it, so a contact called again gets the same version. Hashing a stable ID does this without storing anything:

```python Pick a version per contact theme={"system"}
import hashlib
import os

import requests

VERSIONS = [
    ("0c6a5e0e-7b8f-4d1a-9f2e-3a4b5c6d7e8f", 70),  # control, 70%
    ("9e8d7c6b-5a4f-4e3d-8c2b-1a0f9e8d7c6b", 30),  # challenger, 30%
]

def version_for(contact_id: str) -> str:
    bucket = int(hashlib.sha256(contact_id.encode()).hexdigest(), 16) % 100
    for version_id, weight in VERSIONS:
        if bucket < weight:
            return version_id
        bucket -= weight
    return VERSIONS[-1][0]

requests.post(
    "https://api.bolna.ai/call",
    headers={"Authorization": f"Bearer {os.environ['BOLNA_API_KEY']}"},
    json={
        "agent_id": "123e4567-e89b-12d3-a456-426655440000",
        "agent_version_id": version_for("+919876543210"),
        "recipient_phone_number": "+919876543210",
    },
)
```

A version that doesn't exist, belongs to another agent or was deleted returns `404` with `Agent version not found.`

***

## Split a batch

A batch can split its contacts across two to four versions. Each contact is assigned a version when the batch is uploaded, in proportion to the percentages you set, and its retries keep that version.

> [tab] Dashboard
>
> On the **Batches** page, select your agent and click **Upload Batch**. Turn on **A/B test versions**, then pick a version and a traffic share for each variant. See [Test agent versions against each other](https://www.bolna.ai/docs/outbound/batch-calling.md#test-agent-versions-against-each-other).
>
> ![Upload Batch dialog with A/B test versions turned on](https://mintcdn.com/bolna-54a2d4fe/dmTdbxeZGEzAsBts/images/batch-calling/upload-batch-ab-test.png?fit=max\&auto=format\&n=dmTdbxeZGEzAsBts\&q=85\&s=6bd710b43f6d841df2bca61cdb50bf00)
>
> Upload Batch dialog with A/B test versions turned on
>
> Each row is a variant: choose its version and its share of the batch's contacts. The shares must add up to 100%, and **Split evenly** resets them to equal shares.
>
> ![A/B test versions with v1 and v2 at 50% each](https://mintcdn.com/bolna-54a2d4fe/dmTdbxeZGEzAsBts/images/batch-calling/ab-test-versions-split.png?fit=max\&auto=format\&n=dmTdbxeZGEzAsBts\&q=85\&s=e951686b96c9c5f0dff14b8d0c5c5303)
>
> Two variants, the live v1 and v2, each getting 50% of the contacts

> [tab] API
>
> Pass `version_split` on [Create batch](https://www.bolna.ai/docs/api-reference/batches/create.md#split-a-batch-across-agent-versions):
>
> ```bash theme={"system"}
> curl https://api.bolna.ai/batches \
>   -H "Authorization: Bearer $BOLNA_API_KEY" \
>   -F "agent_id=123e4567-e89b-12d3-a456-426655440000" \
>   -F "file=@recipients.csv" \
>   -F 'version_split={"arms":[{"agent_version_id":"0c6a5e0e-7b8f-4d1a-9f2e-3a4b5c6d7e8f","weight":70},{"agent_version_id":"9e8d7c6b-5a4f-4e3d-8c2b-1a0f9e8d7c6b","weight":30}]}'
> ```
>
> Weights are whole numbers from 1 to 99 that add up to 100, and each arm needs a different version.

All versions in the split need the same telephony provider as the live version, and batches with a workflow can't be split.

***

## Split a workflow agent node

In a [workflow](https://www.bolna.ai/docs/workflows/introduction), an agent node can split its calls across two or more versions of its agent, by weight. Turn on **A/B test versions** in the node's **A/B test** section, or set `version_split` in the node's config: the same `arms` a batch takes, plus `sticky`.

* **Keep each contact on one version**, on by default, gives a contact the same version on retries and repeat visits to the node.
* Each arm follows its version by name, so edits saved into that version reach the test.
* Versions can use different telephony providers; each call uses the provider of the version that runs.
* The campaign page compares the versions, including how each contact's journey ended.

See [A/B test agent versions in workflows](https://www.bolna.ai/docs/workflows/ab-testing.md) for the settings, the rules and what happens when a version changes.

***

## Compare the results

Every call records the version it ran on, however it was started.

* **Dashboard:** in [call history](https://www.bolna.ai/docs/agent-setup/call-history), select the agent and use the **Version** filter and column. **Download Records** includes an `agent_version_id` column.
* **API:** filter [Get agent executions](https://www.bolna.ai/docs/api-reference/executions/get_executions.md#filter-by-agent-version) with `agent_version_id`. Each execution also returns its `agent_version_id`.
* **Workflows:** a campaign's page compares the versions of each A/B-tested agent node, journey outcomes included. See [Compare the versions](https://www.bolna.ai/docs/workflows/ab-testing.md#compare-the-versions).

Compare outcomes such as answer rate, call duration, cost and your [extracted data](https://www.bolna.ai/docs/agent-setup/analytics-tab.md) between versions.

***

## Pick a winner

Make the winning version live with **Make live** in the version switcher, or with [Set current version](https://www.bolna.ai/docs/api-reference/agent/v2/versions/set_current.md). From then on, every call that doesn't ask for a version, including inbound calls, uses it. A workflow agent node keeps splitting its calls until you turn its A/B test off and publish; it then calls the live version.

***

## FAQ

> [accordion] What can I A/B test on a voice agent?
>
> Anything saved in an agent version: the prompts, the welcome message, the voice, the LLM and transcriber models, and the other call settings. Change one thing per version so you know which change made the difference.

> [accordion] Does an A/B test change my live agent?
>
> No. Calls that don't ask for a version, including all inbound calls, keep running on the live version. A version only takes over once you make it live.

> [accordion] How many versions can I test at once?
>
> A batch splits across 2 to 4 versions, and a workflow agent node across 2 or more, with no upper limit. With individual calls there's no limit either, because your code picks the version for each call.

> [accordion] Can I A/B test inbound calls?
>
> No. Inbound numbers always answer with the live version. To compare versions on inbound traffic, make one version live for a period, then the other, and compare the two periods.

> [accordion] Do retries and repeat calls use the same version?
>
> In a batch, yes: each contact gets its version when the batch is uploaded, and its retries keep it. On a workflow agent node, yes while **Keep each contact on one version** is on, as it is by default: retries and repeat visits to the node use the same version. With it off, every visit picks again. With individual calls, it's up to your code, so assign each contact one version and reuse it, as in the [example above](#split-individual-calls).

> [accordion] Can I delete a version while it's being tested?
>
> Not while it's in the split of a batch that hasn't finished, or while a workflow call that has been accepted but hasn't connected yet is waiting on it. [Delete agent version](https://www.bolna.ai/docs/api-reference/agent/v2/versions/delete.md) returns `409` with `This version is in an A/B test on a batch that hasn't finished. Stop or finish that batch first.` until that batch completes or is stopped, and with `This version has workflow calls waiting to be placed. Take it out of the workflow and let those calls connect first.` until those workflow calls connect. The live version can't be deleted either, so make another version live first. Once a version in a workflow agent node's split is deleted, new calls on its arm run the live version and the workflow can't be published until the node is fixed, see [Deleting a version in a test](https://www.bolna.ai/docs/workflows/ab-testing.md#deleting-a-version-in-a-test). Calls that already ran on a deleted version keep their transcripts and still show it as their version.

> [accordion] Can I A/B test a batch that uses a workflow?
>
> Not yet. A batch that runs with a workflow attached can't take a version split. This is separate from [workflow campaigns](https://www.bolna.ai/docs/workflows/campaigns.md): to A/B test the calls a workflow places, [split its agent node](#split-a-workflow-agent-node).

***

## Next steps

> [card] [Agent versions](https://www.bolna.ai/docs/agent-setup/agent-versions.md)
>
> Save, edit and manage versions

> [card] [Batch calling](https://www.bolna.ai/docs/outbound/batch-calling.md)
>
> Upload and run batches

> [card] [Workflow A/B tests](https://www.bolna.ai/docs/workflows/ab-testing.md)
>
> Split a workflow agent node's calls across versions


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.