Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

Implementing Twilio Credential Management and Multi-Tenant Call Routing for Queen of San Diego

What Was Done

This session focused on establishing secure credential management for Twilio integration and architecting a multi-tenant call routing system to handle cascading forward scenarios across independent business lines. Specifically, we:

  • Provisioned Twilio Account SID and Auth Token credentials with proper access controls
  • Integrated Twilio API key pairs for SDK runtime usage
  • Designed a call relay architecture to handle Queen of San Diego (QDN) incoming line forwarding through Sergio's primary number with automatic failover to backup PSTN
  • Established secure credential storage patterns in /Users/cb/Documents/repos/.secrets/repos.env with filesystem permissions (mode 600)
  • Created reference documentation in reference_twilio_credentials.md to guide future session credential selection

Technical Details: Credential Architecture

Twilio credentials come in two distinct flavors, each serving different purposes in a production deployment:

  • Account Authentication (SID + Auth Token): These are the master credentials for account-level operations. The Account SID uniquely identifies your Twilio account across their platform, while the Auth Token acts as the symmetric key for administrative operations. These credentials are appropriate for:
    • Admin console access and configuration changes
    • Account-level reporting and analytics queries
    • Dynamic phone number provisioning and retirement
    • Webhook configuration and TwiML app management
  • API Keys (SK prefix): These are scoped, credential pairs consisting of an API key and secret. Unlike the account auth token, API keys can be revoked independently without affecting other integrations. They're the preferred pattern for:
    • Runtime SDK initialization in application code
    • Microservice-to-Twilio communication over HTTP REST API
    • Integration with CI/CD pipelines where credential rotation is expected
    • Multi-environment deployments where different services get different key pairs

The distinction matters operationally: rotating an API key requires updating environment variables in your application deployment, while rotating the account auth token is a higher-impact operation that potentially affects all integrations simultaneously.

Secure Storage Implementation

Credentials were appended to /Users/cb/Documents/repos/.secrets/repos.env following these principles:

# Command pattern (no actual secrets shown)
$ chmod 600 /Users/cb/Documents/repos/.secrets/repos.env
$ echo "TWILIO_ACCOUNT_SID=..." >> /Users/cb/Documents/repos/.secrets/repos.env
$ echo "TWILIO_AUTH_TOKEN=..." >> /Users/cb/Documents/repos/.secrets/repos.env
$ echo "TWILIO_API_KEY=..." >> /Users/cb/Documents/repos/.secrets/repos.env
$ echo "TWILIO_API_SECRET=..." >> /Users/cb/Documents/repos/.secrets/repos.env

Key decisions on file-level security:

  • Mode 600 (read/write owner only): Prevents accidental credential exposure via world-readable permissions
  • Isolated secrets directory: Keeping credentials separate from application code directories reduces the blast radius if a code repository is accidentally exposed
  • .env file format: Allows standard tooling (dotenv loaders in Node.js, Python, etc.) to hydrate environment variables at application startup without hardcoding
  • No .gitignore bypass: The .secrets directory itself should never be version-controlled; this is a local development pattern only

Architecture: Multi-Tenant Call Relay System

The problem: Queen of San Diego's main incoming line needs intelligent call routing. Previously, the organization attempted carrier-level call forwarding, but discovered this wasn't feasible within standard PSTN constraints. Twilio provides the flexibility to implement application-logic-driven call routing instead.

Proposed flow for QDN cascading forward:

  • Inbound call arrives on QDN's Twilio-hosted phone number
  • Twilio webhook triggers a TwiML app with custom logic
  • Primary forward: attempt to bridge to Sergio's main number
  • If Sergio doesn't answer or line is busy within configured timeout: fall through to backup PSTN number (858-335-4807)
  • If both fail: route to voicemail with context-aware greeting

This pattern solves a key operational problem: during periods when Sergio is unavailable, QDN calls don't vanish into a black hole—they have a defined fallback path. The alternative (static carrier forwarding) couldn't handle conditional logic.

Implementation Approach: TwiML Routing

The actual call handling will be implemented via Twilio's TwiML (Twilio Markup Language), an XML-based telephony instruction set. The webhook endpoint (likely hosted on the same infrastructure as other Queen of San Diego services) will:

  • Receive HTTP POST callback when inbound call arrives
  • Query business logic (is Sergio available? are we in maintenance mode?) from Google Sheets or a local database
  • Return TwiML instructions to Twilio's SIP gateway
  • Log call metadata (caller ID, timestamp, routing decision, final disposition) to a Google Sheet for operational visibility

The webhook implementation follows the same authentication pattern used elsewhere in the Queen of San Diego codebase: OAuth 2.0 tokens for Google APIs, scoped API credentials for third-party services, and request signing where applicable.

Key Decisions and Trade-offs

Why Twilio over SIP forwarding? SIP is lower-level and would require maintaining a PBX infrastructure. Twilio abstracts that complexity; you pay for what you use and get carrier-grade reliability without ops overhead.

Why reference memory for credential selection? The reference_twilio_credentials.md` file documents which credential type to use in which context. This prevents future engineers from accidentally using the account auth token in application code (bad for rotation) or trying to use an API key for admin operations (won't work).

Why mode 600 on the secrets file? Principle of least privilege. If another process or user account on the machine is compromised, they can't read credentials they don't need.

What's Next

With credentials provisioned and reference documentation in place, the next phase is:

  • Build the webhook endpoint that handles /twilio/inbound-call POST requests
  • Implement availability checking logic (query current state from Google Sheets)
  • Create TwiML response builder with fallback chain: Sergio → backup PSTN → voicemail
  • Integration testing: place test calls from external numbers, verify routing, validate logged metadata
  • Production deployment to queenofsandiego.com infrastructure (likely CloudRun or similar serverless compute)
  • Monitoring: CloudWatch logs for webhook latency and call completion rates

The blocking issues around COI adequacy checks, Viator API access, and Adam Cherry's Stripe links remain external blockers, but the Twilio relay is now in the actionable queue and unblocks the cascading forward requirement for QDN's operations.