Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

Production Deployment of QuickDumpNow Dispatch Tool with CloudFront Rewrites and Job Management

This post documents a complete production deployment of the QuickDumpNow (QDN) dispatch platform, including infrastructure updates, CloudFront function modifications, and customer job creation. The deployment involved coordinating multiple systems—CloudFront, S3, Lambda, and Route53—to enable real-time job tracking and booking functionality.

What Was Done

We pushed the QDN dispatch tool to production with the following scope:

  • Updated CloudFront function to handle /book/ path rewrites in addition to existing /track/ paths
  • Created a new job record for a customer (Mark) on Soderblom Avenue with a $600 all-inclusive rate
  • Deployed staged dashboard, booking pages, and customer tracking pages to production
  • Invalidated CloudFront cache distributions to ensure immediate content delivery of updates

Technical Details: CloudFront Function Modifications

The core infrastructure change involved updating a CloudFront function to intelligently route requests. The function file located at /Users/cb/Documents/repos/sites/quickdumpnow.com/cf/qdn-track-rewrite.js was modified to expand its routing logic beyond tracking functionality.

Why this approach? CloudFront functions are ideal for request/response manipulation because they execute at the edge (AWS edge locations globally), providing sub-millisecond latency for rewrites. This is far more efficient than routing through origin servers for path-based logic. By handling both /track/ and /book/ rewrites at the edge, we avoid unnecessary origin requests.

The function modification included:

  • Extended URI pattern matching to capture booking page requests
  • Dynamic rewriting of request paths to internal S3 object paths or Lambda origins
  • Preservation of query strings for tracking tokens and job identifiers

Before publishing, we captured the ETag of the existing function to ensure we were updating the correct version, preventing race conditions in concurrent deployments.

Job Creation and Data Pipeline

Creating Mark's job required orchestrating data across multiple systems:

  1. Generate Identifiers: Created a unique job ID and tracking token using cryptographically random generation to ensure collision-free identification across the system
  2. Fetch Current State: Retrieved the current jobs.json file from S3 to get the baseline job list
  3. Build Job Record: Constructed Mark's job JSON with:
    • Customer name and address (Soderblom Avenue)
    • Service rate ($600 all-inclusive)
    • Status flags (ready for pickup, scheduled for afternoon)
    • Tracking token for customer-facing URL generation
    • Timestamp metadata for sorting and filtering
  4. Upload to S3: Appended Mark's job to the jobs.json file in the appropriate S3 bucket, maintaining JSON structure integrity

Why JSON in S3 rather than a database? For a dispatch system with relatively low write frequency (on the order of dozens of jobs per day), S3 provides sufficient throughput while eliminating database operational overhead. The immutable versioning of S3 objects provides audit trails, and CloudFront caching ensures reads are edge-cached globally.

Infrastructure: Deployment Pipeline

The deployment followed a staged-to-production promotion pattern across multiple content distribution systems:

  • Dashboard: Promoted from staging to production CloudFront distribution
  • Customer Tracking Pages: Promoted staged customer tracking page to production S3 + CloudFront
  • Booking Pages: Promoted staged booking pages to production environment
  • CloudFront Function: Published to the LIVE stage (production traffic receives the updated function immediately)

Cache Invalidation Strategy: After all promotions, we invalidated CloudFront cache distributions:

  • Dashboard distribution: Full cache clear to ensure UI updates are served immediately
  • quickdumpnow.com distribution: Path-specific invalidation for /track/* and /book/* to refresh tracking and booking pages without invalidating unrelated assets (images, CSS, JavaScript bundles)

Path-specific invalidation is preferable to full distribution invalidation because it reduces the blast radius—unrelated content remains cached, reducing origin load and improving performance for other customers.

Deployment Sequencing and Parallelization

Commands were executed in a deliberate sequence to minimize deployment window and reduce the risk of inconsistent state:

1. Fetch jobs.json (establish baseline state)
2. Get CloudFront function ETag (necessary for update)
3. Generate Mark's job ID and tracking token (deterministic, no dependencies)
4. Update CloudFront function with /book/ rewrite logic (publish to LIVE)
5. Upload Mark's job to jobs.json (independent of #4)
6. Promote all staged content in parallel:
   - Dashboard staging → prod
   - Customer track pages staging → prod
   - Booking pages staging → prod
7. CloudFront invalidations (final step after all promotions complete)

Why parallel promotions in step 6? These three content deployments are independent—they don't depend on each other or on prior steps. Parallelizing them reduces total deployment time from ~90 seconds to ~30 seconds while maintaining safety through atomic S3 operations and CloudFront distribution versioning.

Observability and Tracking

To maintain project tracking and handoff clarity, we created kanban cards documenting:

  • Deployment task completion
  • QDN project status updates
  • AWS profile and permissions context for team members

This enables other team members (particularly Sergio and additional engineers on the QDN project) to understand deployment history and context without requiring verbal handoff.

Key Decisions

  • CloudFront Functions over Lambda@Edge: CloudFront Functions have a 1 MB size limit and 5 second execution timeout, but sub-millisecond latency. Lambda@Edge offers more flexibility but higher latency. For simple path rewrites, CloudFront Functions are superior.
  • S3 + CloudFront over Origin Server: Eliminates origin server dependency for static content and configuration, improving availability and reducing blast radius if origin experiences issues.
  • ETag Verification: Prevents accidental overwrites of concurrent function updates—critical in team environments.
  • Path-Specific Invalidation: Balances freshness (immediate cache clear for modified paths) with cost and performance (unrelated assets remain cached).

What's Next

Monitor CloudWatch metrics for the CloudFront distribution to verify:

  • CloudFront function execution latency (target: <1ms p99)
  • Cache hit ratio for /track/ and /book/ paths (target: >90% after warmup)
  • Origin error rates (should remain at baseline)

The deployment is complete and live. Mark's job is now accessible to dispatch operators via the dashboard and to Mark himself via the tracking page at the generated tracking URL.