Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

```html

Building a Multi-Tenant SMS Relay for QuickDumpNow: Twilio Integration & CloudFront Tracking

What Was Done

This session focused on completing the infrastructure foundation for QuickDumpNow (QDN), a waste management dispatch platform. The work centered on three critical areas: Twilio credential integration for SMS relay cascading, a customer-facing job tracking page deployed via CloudFront, and Lambda-based APIs to support real-time job status and message persistence.

The core business requirement driving this work: QDN's dispatch workflow needs to forward incoming SMS messages from a primary Twilio number through secondary carriers (a main dispatcher number and a backup number) when the primary line is busy or unreachable. This cascading behavior prevents missed job requests in a time-sensitive, field-service environment.

Technical Details: Architecture & Implementation

Credential Management & Environment Setup

Twilio credentials were integrated into the shared repository secrets file at /Users/cb/Documents/repos/.secrets/repos.env with restrictive file permissions (mode 600). Two credential types were stored:

  • Account Auth Token: Used for administrative operations (SMS sending, webhook verification, phone number management) via the Twilio REST API
  • API Key Pair: Preferred for SDK runtime operations within Lambda functions, providing narrower scope and easier rotation without impacting account-level access

This separation reflects a security best practice: administrative credentials are isolated from application runtime credentials. Future Lambda deployments reference the API key pair, while administrative tasks (Twilio CLI, webhook setup) use the account token.

Lambda Function: Message Persistence & Job State

The Lambda function at /Users/cb/Documents/repos/sites/dashboard.quickdumpnow.com/lambda/lambda_function.py was extended to handle four new API endpoints:

  • POST /api/jobs — Create a new dispatch job with initial status (pending)
  • GET /api/jobs/{job_id} — Retrieve job details and current status
  • POST /api/jobs/{job_id}/messages — Append a customer notification message to job history
  • GET /api/jobs/{job_id}/messages — Fetch message log for a job (used by tracking page)

Each endpoint includes CORS headers to allow cross-origin requests from the public tracking page. The Lambda integrates with DynamoDB to persist job state and message logs; a seed file at /tmp/maintenance_seed.json was prepared to bootstrap the table with test data before production deployment.

Customer Tracking Page & CloudFront Delivery

A public-facing tracking page was created at /Users/cb/Documents/repos/sites/quickdumpnow.com/track/index.html. This single-page application:

  • Accepts a job ID as a URL parameter (e.g., https://quickdumpnow.com/track/?job=J1234)
  • Fetches job status and message history from the Lambda API endpoints
  • Displays real-time updates (job status, ETA, service completion, SMS notification log) without requiring customer authentication
  • Uses client-side polling (5-second intervals) to refresh status; could be upgraded to WebSocket or Server-Sent Events for lower latency

The page was deployed to the QuickDumpNow S3 bucket and served through CloudFront distribution d3xqxyz... (exact ID withheld). A CloudFront function was created to rewrite /track requests to /track/index.html, preserving query parameters and enabling clean URLs.

API Gateway & Route Integration

Four new routes were added to the QDN API Gateway stage to expose the Lambda endpoints. Each route includes:

  • Explicit HTTP method restriction (GET vs POST)
  • Request/response model validation
  • OPTIONS endpoints for CORS preflight
  • CloudWatch logging for audit and debugging

The API URL structure is https://api.quickdumpnow.com/jobs and https://api.quickdumpnow.com/jobs/{job_id}/messages, consumed by both the tracking page and downstream webhook handlers.

Infrastructure & Key Decisions

Why API Key Pair Over Account Token

The Twilio API key pair is narrower in scope—it cannot change account settings, delete numbers, or modify billing—making it safer for long-lived runtime credentials in Lambda. Account tokens are reserved for one-time administrative tasks. This reduces the blast radius if credentials are accidentally logged or exposed.

CloudFront Function for URL Rewriting

Rather than managing index.html rewrites in S3 static website config (which is error-prone and less flexible), a CloudFront function intercepts requests to /track and rewrites them to /track/index.html at the edge. This is faster than origin-based redirects and maintains the original URL in the browser, preserving query parameters without round-trips.

DynamoDB Over RDS for Job State

Job and message data are stored in DynamoDB rather than a relational database. The rationale: dispatch jobs are immutable append-only logs (new messages are added, old records never updated), and DynamoDB's stream-based triggers can fuel downstream webhooks (SMS relay, dispatcher notifications) without polling. Query patterns are simple (job ID lookup), favoring NoSQL.

Stateless Lambda + API Gateway

Lambda functions remain stateless; all state lives in DynamoDB. This enables horizontal scaling without session affinity and simplifies disaster recovery. API Gateway provides a stable public URL, load balancing across Lambda instances, and automatic cold-start mitigation through provisioned concurrency (configured for baseline traffic spikes during job creation).

Deployment & Testing

Deployment followed this sequence:

  1. Update Lambda function code locally and test with mock events
  2. Package and deploy via AWS CLI to the qdn-data-crud function
  3. Verify new routes in API Gateway; test with curl to confirm response shape
  4. Deploy tracking page to S3 and invalidate CloudFront cache
  5. Create/attach CloudFront function and publish to the distribution's default behavior
  6. Smoke test: create a test job via API, fetch it back, verify message persistence

Commands used (no credentials shown):


# Package and deploy Lambda
aws lambda update-function-code --function-name qdn-data-crud \
  --zip-file fileb://lambda_function.zip

# Invalidate CloudFront cache after updating tracking page
aws cloudfront create-invalidation --distribution-id D3XQXYZ \
  --paths "/*"

# Test API endpoint
curl https://api.quickdumpnow.com/jobs/J1234/messages -H "Content-Type: application/json"

What's Next

With the tracking page and API foundation live, the next phase is building the Twilio relay logic. This involves:

  • Configuring a Twilio webhook to receive inbound SMS on the primary QDN number
  • Writing a Lambda function that parses the SMS, creates a new job record, and attempts to forward to the dispatcher via Twilio's API
  • Implementing casc