CuePilot
Real-time voice AI that suggests responses to customer support agents during live calls.
Overview
Customer support agents lose time thinking about the right response during complex calls. CuePilot intercepts audio in real-time, transcribes via Whisper, and surfaces the optimal response within 200ms — so agents focus on the human, not the answer.
The WebSocket pipeline streams audio from the agent's browser tab, processes it through a FastAPI backend, and pushes LLM suggestions to a live UI. Agents can read, adapt, or copy the suggestion with one click.
What Was Built
- →Real-time browser audio capture via MediaRecorder API
- →WebSocket streaming pipeline (browser → FastAPI)
- →Whisper speech-to-text with incremental transcription
- →LLM response generation under 200ms average latency
- →Agent suggestion dashboard with copy/adapt actions
- →Response history and session analytics
- →Multi-agent team mode (parallel copilot instances)
The Challenge
Achieving sub-200ms suggestion latency end-to-end across the audio capture → transcription → LLM → UI delivery chain while keeping the pipeline stable under concurrent users.
The Solution
Whisper runs on a GPU-enabled FastAPI worker with connection pooling. The LLM layer uses Groq for inference speed. WebSocket connections are multiplexed per session, not per agent, reducing overhead.
Have a similar project?
Tell us what you need. We'll come back with a scoped plan tailored to your requirements.
Start a Project