Live demo benchmark
How quickly the guide responds, how often it finishes the task, how it copes with interruptions, and what a minute of conversation costs us. Measured on a sample app, with the method published first.
Results
| Measure | Statistic | Result | Runs |
|---|---|---|---|
| Press to ready | Median (p50) | Pending | Pending |
| Slow end (p95) | Pending | ||
| Time to first word | Median (p50) | Pending | Pending |
| Slow end (p95) | Pending | ||
| Time to first action | Median (p50) | Pending | Pending |
| Slow end (p95) | Pending | ||
| Task success | Rate | Pending | Pending |
| Interruption recovery | Rate | Pending | Pending |
| Cost per call-minute | Mean | Pending | Pending |
What each measure means
Timings run from what the visitor does to what they experience, so they include the network and the browser, not only our servers.
- Press to ready
- From pressing the talk button to the guide being ready to listen, including the microphone prompt.
- Time to first word
- From the moment the visitor stops speaking to the first audio of the guide’s reply.
- Time to first action
- From the end of a request to the first completed action on the page: a highlight, a click or a filled field.
- Task success
- Share of scripted tasks completed, checked by reading the app’s state afterwards rather than trusting the guide’s word.
- Interruption recovery
- Share of runs where the visitor cuts in mid-reply and the guide stops, listens and carries on correctly.
- Cost per call-minute
- Model and voice provider charges for the runs divided by call minutes, reconciled against the provider bill.
How we run it
- An owned sample web app with several pages, reset to the same state before every run.
- A fixed script of visitor requests, spoken from recordings so every run hears the same audio.
- Runs split into cold starts (first call after idle) and warm calls, reported separately.
- Device, browser and network recorded for every run. Results are labeled with them.
- Failed and abandoned runs stay in the data. Nothing is dropped to improve a number.
What this won’t tell you
It measures Showalong on our own sample app. Your app will differ: bigger pages, slower networks and longer flows all change the numbers. We will add results from customer apps only when customers agree to share them.
It does not compare us with other products. We haven’t run their tools through the same tasks, so we won’t publish numbers for them.
Cost per call-minute is our cost, not your price. Your price is on the pricing page.