Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

```html

Building a Multi-Tenant Tracking & Notification Pipeline: QDN's Journey from Manual Dispatch to Automated Customer Updates

What Was Done

Over this development session, we completed the foundational infrastructure for QuickDumpNow's (QDN) customer notification system—a critical piece missing from their four-trailer fleet tracking workflow. The goal: automatically notify customers of job status changes via SMS without manual dispatcher intervention.

The work spanned three major areas:

  • Twilio integration: Provisioned account, API keys, and SMS relay logic
  • Lambda-based data pipeline: Extended the qdn-data-crud Lambda to handle message persistence and job-status-triggered notifications
  • Frontend tracking page: Built a public-facing /track endpoint so customers can check job status in real time

Technical Details

Twilio Account & Credential Management

We provisioned a Twilio account and extracted two credential pairs:

  • Account SID + Auth Token: Used for administrative API calls (Twilio REST API v1), credential management, and account inspection. Stored in /Users/cb/Documents/repos/.secrets/repos.env as TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN.
  • API Key + Secret (Main): Preferred for runtime SDK operations in Lambda and application code. Also persisted in repos.env as TWILIO_API_KEY and TWILIO_API_SECRET.

Why two pairs? The API key/secret pattern allows scope-limiting at the Twilio level—you can revoke a single key without rotating your master account credentials. For Lambda runtime, this is safer and operationally cleaner.

All credentials were appended to repos.env with file mode 600 (readable/writable by owner only) to prevent accidental exposure in logs or version control.

Lambda Message Pipeline: qdn-data-crud

The existing qdn-data-crud Lambda (/Users/cb/Documents/repos/sites/dashboard.quickdumpnow.com/lambda/lambda_function.py) was extended to:

  1. Accept new routes for message persistence: We added four new API Gateway routes to handle job-status updates, token lookups, and message log queries.
  2. Trigger SMS on job state changes: When a job transitions to a trackable state (e.g., "En Route," "Arrived," "Complete"), the Lambda invokes the Twilio SDK to send an SMS to the customer phone number stored in the job record.
  3. Log all outbound messages: Each SMS is persisted to DynamoDB (table name referenced in the Lambda environment) with timestamp, job ID, customer phone, message body, and Twilio SID for audit/retry purposes.

The message template logic checks job type and status to build context-aware copy. For example:

if job_status == "en_route":
    message = f"Your {job['service_type']} job (ID: {job['id']}) is on the way. Track it: {tracking_url}"
elif job_status == "complete":
    message = f"Your job is complete. Total: ${job['total']}. Invoice: {invoice_url}"

This avoids generic messages and gives customers actionable information immediately.

Frontend Tracking Page

Created /Users/cb/Documents/repos/sites/quickdumpnow.com/track/index.html—a public-facing page that accepts a job token (generated at job creation) and displays real-time status without requiring login.

The page makes CORS-enabled calls to the new Lambda endpoints:

GET /jobs/token/{token}  → returns job details + latest status
GET /jobs/{id}/messages → returns SMS log for customer review

We created a CloudFront function to rewrite incoming /track?token=xyz requests to the correct backend path, avoiding client-side routing complexity. The function is attached to the default behavior of the quickdumpnow.com CloudFront distribution and publishes cache-busting headers for the track/index.html object.

Infrastructure Changes

  • API Gateway: Added 4 new routes to the existing QDN API, each with OPTIONS method for CORS preflight. No new Gateway deployment—routes attach to existing qdn-api stage.
  • Lambda: Updated qdn-data-crud function code and added two environment variables: TWILIO_API_KEY and TWILIO_API_SECRET. Execution role already had DynamoDB write permissions; no IAM changes needed.
  • DynamoDB: Assumed existing qdn-messages table (or created if absent) with partition key job_id and sort key sent_at.
  • S3 + CloudFront: The tracking page HTML is deployed to the QDN S3 bucket (same bucket serving the main site), and the CloudFront function is published to the quickdumpnow.com distribution.
  • Twilio: Provisioned a Twilio phone number (assigned to the account) for outbound SMS. This number is stored as an environment variable in the Lambda.

Key Decisions & Rationale

Why Lambda instead of a separate microservice? QDN already runs on Lambda for API logic. Adding notification logic to the same function keeps operational surface minimal—one function to monitor, one set of logs, no cross-service orchestration. As volume grows, we can split into a separate notification Lambda later without breaking the data pipeline.

Why persist messages to DynamoDB? Customers and dispatchers need audit trails. Relying only on Twilio's API for history is risky if the account is compromised or if we need to debug failed sends. DynamoDB gives us full control and integrates seamlessly with the existing job data model.

Why CloudFront function for /track rewrites? The tracking page is public and stateless, so it benefits from edge caching. The CloudFront function handles URL transformation at the edge without burdening the origin, reducing latency for customers checking status from the road.

API Key over Account Token in Lambda: Scope is a security principle. If a Lambda is compromised, an attacker with only an API key can't rotate credentials or access billing—they can only send SMS from that key's quota. The account token would be catastrophic.

Testing & Validation

We ran end-to-end smoke tests:

  1. Created a test job in the dashboard.
  2. Looked up the job token via the Lambda's GET /jobs/token/{token} endpoint.
  3. Updated the job status to "en_route" and verified an SMS was sent to the test phone number.
  4. Checked the DynamoDB log to confirm the message row was persisted with correct metadata.
  5. Accessed the public tracking page with the token and verified status display matched