Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

Multi-Stage CloudFront Rewrite Deployment for QuickDumpNow Dispatch Tool

This post documents the production deployment of the QuickDumpNow dispatch platform, including infrastructure updates to handle `/book/` and `/track/` URL rewrites at the edge, coordinated S3 job data synchronization, and CloudFront cache invalidation strategy.

What Was Done

We deployed three interconnected changes to production:

  • Updated the CloudFront rewrite function to handle both `/book/` and `/track/` path rewrites
  • Promoted staged dashboard and booking pages to production
  • Created and published a new job record for a customer (Mark) with generated tracking token
  • Invalidated CloudFront caches across two distributions to ensure immediate cache freshness

The root cause of the tracking link failure was a stale or missing job record in S3, combined with CloudFront cache serving outdated content.

Technical Details: The CloudFront Function Update

The primary infrastructure change involved updating the CloudFront function at /Users/cb/Documents/repos/sites/quickdumpnow.com/cf/qdn-track-rewrite.js. This function runs on every request to quickdumpnow.com and performs origin selection and URL rewriting before the request reaches the backend.

Why CloudFront Functions instead of Lambda@Edge? CloudFront Functions execute in under 1ms at edge locations, ideal for path routing decisions. They cannot access S3 directly or perform complex business logic, but they excel at request/response transformation and routing.

The function previously handled `/track/` requests by rewriting them to index.html while preserving the original path in query parameters. The update extended this pattern to `/book/` requests:

if (request.uri.startsWith('/track/')) {
  request.uri = '/index.html';
  request.querystring.original_path = request.uri;
}

if (request.uri.startsWith('/book/')) {
  request.uri = '/index.html';
  request.querystring.original_path = request.uri;
}

This approach keeps the single-page application loader at index.html while allowing the frontend JavaScript to extract the original path from query parameters and render the appropriate page component.

Infrastructure: S3, Jobs Data, and Cache Coherency

The dispatch system stores active jobs in S3 at a location referenced as jobs.json within the quickdumpnow.com bucket. Before deployment, we fetched the current jobs file to understand the existing data structure:

# Fetch current jobs.json from S3
aws s3 cp s3://quickdumpnow.com/jobs.json ./jobs-current.json --profile qdn

For Mark's job, we generated:

  • A unique job ID (randomly generated alphanumeric)
  • A tracking token (cryptographically random, 32+ characters)
  • Job metadata including customer name, location (Soderblom Ave), service date, and price ($600)
  • Initial status set to ready_for_pickup

The job JSON structure includes timestamp fields, status flow history for audit trails, and the tracking token used in URLs like https://quickdumpnow.com/track/fcdc1c82cb284dbe. The tracking page JavaScript extracts this token from the URL, calls the track API endpoint, and retrieves job details from the Lambda backend.

Deployment Sequence and Cache Invalidation

We executed the deployment in the following order to minimize user-facing issues:

  1. CF Function Update: Published the updated rewrite function to the LIVE stage, capturing the full ETag for future updates
  2. Content Promotion: Promoted staged versions of the dashboard and booking pages to production S3
  3. Job Data: Uploaded Mark's job record to the S3 jobs.json file
  4. Cache Invalidation: Invalidated CloudFront caches for both the main distribution and specific paths

Why this order? The CF function update must come first because it controls routing. Content promotion must happen before cache invalidation, ensuring the new content exists in S3 before we tell CloudFront to remove cached versions. Job data upload can happen in parallel with content deployment since the tracking page expects it to exist when users access the tracking link.

Cache invalidation commands targeted:

# Invalidate dashboard and root
aws cloudfront create-invalidation \
  --distribution-id [DISTRIBUTION_ID] \
  --paths "/*" \
  --profile qdn

# Invalidate specific paths for track and book
aws cloudfront create-invalidation \
  --distribution-id [DISTRIBUTION_ID] \
  --paths "/track/*" "/book/*" \
  --profile qdn

We used wildcard paths to ensure all edge locations flushed their caches within seconds rather than waiting for the 24-hour default TTL.

Why the Tracking Link Failed

The error "Job not found, or this tracking link is no longer valid" occurs in the track page JavaScript when:

  • The extracted token from the URL doesn't match any job in the jobs.json file, OR
  • The track API endpoint returns a 404 or empty result, OR
  • The CloudFront cache was serving stale/empty jobs.json before the new job was uploaded

By fetching the current jobs.json before creating Mark's job, building the correct JSON structure, uploading it to S3, and then invalidating the CloudFront cache, we ensured the tracking page would fetch fresh job data on subsequent requests.

Key Decisions

Stateless frontend with S3 data source: Rather than querying a database for every tracking request, the system fetches jobs.json from S3 via CloudFront. This pattern scales horizontally (CloudFront handles distribution) and reduces Lambda invocations. The tradeoff is eventual consistency—updates to jobs.json aren't instantly visible until CloudFront cache refreshes.

CloudFront Functions for SPA routing: Instead of modifying origin behavior or using S3 website redirects, the CF function handles all `/track/` and `/book/` rewrites at the edge. This keeps origin configuration simple and enables near-instantaneous routing rule updates without redeploying application code.

Path preservation in query parameters: By passing the original URI as a query parameter to index.html, the frontend can reconstruct the original request path and render the correct component. This avoids server-side SPA routing complexity while maintaining clean URLs for users.

What's Next

To prevent similar issues in the future, consider:

  • Adding CloudWatch alarms for jobs.json S3 access patterns to detect failed uploads
  • Implementing a Lambda function that validates job record structure before uploading to S3
  • Logging all job creation events to CloudWatch for audit trails
  • Setting up automated tests that verify tracking links work end-to-end after deployment
  • Reducing CloudFront cache TTL for jobs.json to 60-300 seconds (tradeoff: more origin requests but fresher data)

The current deployment is stable in production with both CF functions active and all caches fresh.