Architecture Deep Dive

CVS Outbound Dialer

An automated outbound calling platform built on LiveKit Agents, deployed entirely on-prem.

press space or scroll to begin

$ cat agenda.md

01

What Is It?

Components & design decisions

02

LiveKit Agents

Framework, AgentServer, JobContext

03

Infrastructure

On-prem RKE cluster & services

04

Call Flow & SIP

End-to-end sequence + SIP trunk

05

Conversations

One-way, two-way & AMD

06

Ops

Recording, audit, deploy, TURN

01

What Is It?

The Problem

Automated outbound calls for CVS Health campaigns

One-way notifications (prescription reminders) + two-way interactive conversations (appointment scheduling) with AMD, recording & full audit trails.

# Key Design Decisions

No built-in LLM — The agent's llm_node() calls an external agent API over HTTP. Conversational logic is fully decoupled from the voice pipeline.
Single image, dual deploy — One Docker image runs as either the API server or agent worker, switched by the LIVEKIT env var.
On-prem everything — LiveKit, ASR/TTS, campaign service, event publisher all run on CVS infrastructure. Only Twilio (SIP + TURN) is external.

# Core Components

API

outbound-dialer

FastAPI service. Receives call requests, creates rooms, dispatches agents, initiates SIP.

WKR

outbound-dialer-worker

LiveKit Agent process. Handles real-time voice AI logic via AgentSession.

LK

LiveKit Server

Self-hosted WebRTC server. Routes audio, manages rooms, bridges SIP, runs egress.

SIP

Twilio SIP Trunk

External PSTN gateway. Terminates outbound SIP calls to phone numbers.

02

LiveKit Agents Framework

# LiveKit Agents SDK

The agent worker is built on the LiveKit Agents SDK — a Python framework for building real-time voice AI agents that participate in LiveKit rooms as first-class participants.

AgentServer

The top-level process that connects to the self-hosted LiveKit server over WebSocket. It listens for agent dispatch requests and spawns agent sessions.

src/outbound_dialer/agent/entrypoint.py

JobContext

Provided to the entrypoint when the agent is dispatched. Gives access to the LiveKit room, participant events, and room metadata. Calls ctx.connect() to join (with TURN).

# Agent Architecture

AgentServer

JobContext

AgentSession

OutboundDialerAgent

AgentSession — Manages the real-time voice pipeline. Configured with STT (CVS ASR), TTS (CVS TTS), VAD (Silero), and turn detection (MultilingualModel).
OutboundDialerAgent — Extends livekit.agents.Agent. Overrides stt_node(), tts_node(), and llm_node() to route through CVS services + external API.
Worker → LiveKit — Reads LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET from Vault. Establishes persistent WebSocket. When dispatched, spawns JobContext → calls entrypoint.

03

On-Prem Infrastructure

# System Architecture

RKE Kubernetes — CVS On-Prem

API Server
FastAPI :21121

Agent Worker
LIVEKIT=true :8081

LiveKit Server
wss://…livekit.corp

Egress
→ GCS MP3

↕

Internal CVS Services

CVS ASR
wss://…/listen

CVS TTS
wss://…/speak

Campaign Svc

Event Pub → Kafka

↕

External — Twilio

SIP Trunk
TLS + SRTP

PSTN Gateway

TURN Server
:443 TCP

# Internal Service Map

Service	Protocol	URL
LiveKit Server	WebSocket/gRPC	wss://devservices-colo-west-livekit.corp…
CVS ASR	WebSocket	wss://…speak-colo-west…/v1/listen
CVS TTS	WebSocket	wss://…speak-colo-west…/v1/speak
Campaign Service	HTTPS	https://devservices-colo-west…/campaigns/v1
Event Publisher	HTTPS	https://digital-retail-rx-qa…/kafka/send-message

04

End-to-End Call Flow

# SIP Call Sequence

End-to-end SIP call flow sequence diagram showing Client, API Server, LiveKit, Agent Worker, and Twilio SIP interactions

# Call Flow — Setup

1. API Request — External system calls POST /api/v1/calls/sip with campaignId, phoneNumber, sessionId, and custom data.
2. Campaign Config — API server fetches campaign configuration (messages, AMD settings, agent paths) from internal Campaign Service.
3. Room Creation — LiveKit room created with metadata containing full config, phone number, session ID, and recording flag.
4. Agent Dispatch — Agent dispatched to room. LiveKit notifies worker → spawns JobContext → calls outbound_agent_entrypoint().

# Call Flow — Connection

5. SIP Participant — LiveKit creates SIP participant via SIP API → triggers SIP INVITE to Twilio over TLS + SRTP.
6. Phone Answers — SIP participant joins the room. Agent detects via ctx.wait_for_participant(), starts recording, begins conversation.

Source: src/outbound_dialer/services/call_service.py, sip_service.py

# SIP Telephony Architecture

On-Prem LiveKit Server

LiveKit Room

Agent (worker) audio
SIP Participant audio
Mixed audio (recording)

SIP Bridge

Trunk: cvs-retail-outbound
Address: cvsretailoutboundterminate.pstn.ashburn.twilio.com
From: +12134147078
Transport: TLS
Media: SRTP encrypted

LiveKit SIP Bridge

SIP INVITE (TLS)

Twilio PSTN Gateway

Phone ☎

Credentials stored in Vault · SIPServiceSingleton ensures trunk exists at startup

05

Conversation Engine

# Conversation Routing

SIP Participant Joins Room

Start Egress Recording

campaign.type?

one_way

Play message → hang up

session.say() with
interruptions disabled

two_way

Full voice pipeline

STT → AMD → LLM → TTS
interactive loop

# Two-Way Voice Pipeline

User Speech

STT
CVS ASR

AMD Check

LLM Node
External API

TTS
CVS TTS

Speaker

STT — Deepgram nova-3 via CVS internal ASR (WebSocket)
AMD — Pattern matches first utterance for voicemail phrases
LLM Node — HTTP POST to external agent API, not a built-in LLM
TTS — Deepgram aura-2-helena-en via CVS internal TTS

# Answering Machine Detection

First user utterance (via STT)

Contains voicemail pattern?

No

Continue conversation

Yes → AMD enabled?

HANGUP

Publish amd_hangup
→ End call

LEAVE_MESSAGE

Play voicemail msg
→ Delayed hangup

06

Recording, Audit & Operations

# Egress Recording Architecture

Self-Hosted LiveKit Server

LiveKit Room

Agent (worker) audio track
SIP Participant (phone) audio track

subscribe
all audio →

Egress Service

RoomComposite
audio_only = True

→

MP3 Encode

Mixed audio stream

→

GCS Upload

Bucket: livekit-recordings-nonprod
Path: recordings/{campaign}/{session}/recording.mp3

Auto-stops on participant departure (30s timeout) · Manual stop via stop_and_wait_for_egress()

# Event Audit System

API Server
session events

Agent Worker
conversation events

Event Publisher
HTTPS

Kafka Topics

Event	When Published
conversation_started	Greeting delivered (two-way)
notification_delivered	One-way message played
amd_hangup	Voicemail detected, call hung up
user_hangup	User disconnected early
error_http / error_exception	Agent API or runtime error

# TURN Server Configuration

TURN relays are required because the on-prem LiveKit server needs to route media to/from external Twilio SIP endpoints across the corporate firewall.

Corporate Network

LiveKit Server

on-prem

Agent Worker

on-prem

NAT / Firewall
direct path blocked

↕

Twilio TURN
global.turn.twilio.com:443
TCP · relay-only transport

↕

External — Twilio

SIP Gateway

PSTN Bridge

TRANSPORT_RELAY forced · Credentials fetched from Twilio API, cached 24h · Stored in Vault

# Dual-Mode Deployment

API Server

Port	21121
Health	/health
Env	LIVEKIT not set
Entry	scripts/start.py → FastAPI
Route	/microservices/outbound-dialer/

Agent Worker

Port	8081
Health	/
Env	LIVEKIT=true
Entry	entrypoint.py → AgentServer
Route	/microservices/outbound-dialer-worker/

Single Docker image · docker_entrypoint.py checks LIVEKIT env var to select mode

# Kubernetes Operations

Clusters — RKE on-prem: x42-west-qa, x42-west-prod, x42-east-prod
HPA — QA: 2-4 replicas · Prod: 5-10 replicas · 50% CPU target · 10 TPS/pod
Resources — 3-4 CPU, 4-6Gi memory per pod · PVC for HuggingFace models (Silero VAD, turn detector)
Secrets — HashiCorp Vault → LiveKit keys, Twilio creds, I90 auth, TLS certs
Health — Startup probe: 50 failures × 10s ≈ 8 min (accounts for model loading)

# CI/CD Pipeline

CI — Continuous Integration

Push to branch

Python 3.13 Build

pytest

Docker Build

Push → JFrog

CD — Continuous Deployment

deploy-configs/** change

Detect env

Validate TPS

Helm Deploy

RKE Cluster

API server and worker have separate CD pipelines triggered by different deploy-configs/ paths

Key Takeaways

$ summary

On-prem first — Everything runs on CVS infrastructure. Only external dependency is Twilio.
LLM-decoupled — Conversational logic lives in an external API, not baked into the voice pipeline.
Single-image dual-mode — One Docker image, one codebase, two deployment modes via env var.
Full audit trail — Every call recorded to GCS, every event published to Kafka.

end of presentation