Prerequisites
- Python 3.12+
- Docker (for Postgres + Redis)
- Twilio account with a phone number
- OpenAI API key (or Ollama for local LLM)
- Deepgram API key (free at console.deepgram.com)
- ngrok (for local development)
Setup
What You Just Built
Twilio opens a media-stream WebSocket to/ws/media-stream; the pipeline runs until you hang up, then the call is marked completed.
Manual Setup via API
If you prefer to set things up step by step:Create a project
Create an API key
Create an agent
Publish the agent
Bind a phone number
Configure Twilio webhooks
Set your Twilio number’s webhook URLs:- Voice URL:
https://xxxx.ngrok.io/webhooks/twilio/voice/inbound(POST) - Status URL:
https://xxxx.ngrok.io/webhooks/twilio/status(POST)
Call your number
That’s it — call the number and talk to your agent.Environment Variables
| Variable | Required | Description |
|---|---|---|
DATABASE_URL | Yes | PostgreSQL connection string |
REDIS_URL | Yes | Redis connection string |
TWILIO_ACCOUNT_SID | Yes | Twilio account SID |
TWILIO_AUTH_TOKEN | Yes | Twilio auth token |
OPENAI_API_KEY | Yes* | OpenAI API key. *Not required if using Ollama |
DEEPGRAM_API_KEY | Yes | Deepgram API key |
ELEVENLABS_API_KEY | No | ElevenLabs API key |
CARTESIA_API_KEY | No | Cartesia API key |
ANTHROPIC_API_KEY | No | Anthropic API key (for Claude) |
GOOGLE_API_KEY | No | Google API key (for Gemini Live S2S) |
OPENROUTER_API_KEY | No | OpenRouter API key (multi-model + fallback routing) |
HEYGEN_LIVE_AVATAR_API_KEY | No | HeyGen video avatar (LiveAvatar key, app.liveavatar.com) |
TAVUS_API_KEY | No | Tavus video avatar (platform.tavus.io) |