Benchmarks
Benchmarks and methodology
Our own measurements aren't published yet. They come after the pilot, measured on real jobs with the method below. Until then, this page shows public third-party figures that our design is based on, each with its source. None of them are our results.
Methodology (what we will publish)
- Workload. A fixed, public test set: long-form English and multilingual podcasts and Spaces (10 to 90 minutes), plus a short-clip set under 30 seconds. Embeddings: a public forum corpus.
- Accuracy. Word error rate against human reference transcripts, after standard text normalisation. Embeddings: retrieval recall@10 against the reference model run on our own hardware.
- Throughput. Audio hours per wall-clock hour for a batch of 100 files, measured end to end from submission to the last result.
- Cost. What the buyer pays per audio hour at list price, next to sourced competitor list prices on the same date.
- Distribution, not best case. We report p50 and p95 and include failed and re-run jobs.
Reference figures from public sources
| Finding | Source |
|---|---|
| Whisper large-v3 on 100 consumer-GPU replicas: nearly 8,000 audio hours in about 10 hours for roughly $110 all-in | Salad |
| Clips under 30 seconds yielded about 4x less audio per dollar than long files on the same GPUs | Salad |
| RTX 4090, faster-whisper large-v3: about 14x real time single-stream, 175–250x batched | RunPod |
| CPU-only: a Ryzen 9 5950X needed 8.1 s to encode a 30 s window; an i5-8250U laptop needed 61.5 s (slower than real time) | whisper.cpp #89 |
| Phones lose 15–45% of peak speed within minutes of sustained load | arXiv 2603.23640 |
Market prices on the comparison date
- Groq Whisper large-v3-turbo: $0.04 per audio hour (developersdigest)
- DeepInfra Voxtral-Mini: $0.06 per audio hour (dev.to comparison)
- Salad Transcription API (consumer GPUs): $0.16–0.20 per audio hour (Salad billing docs)
- Deepgram Nova-3 batch: about $0.26 per audio hour (dev.to comparison)
- OpenAI Transcription API: $0.36 per audio hour (dev.to comparison)
Planned First write-up: after the pilot. Current prices.