Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

Multi-Stage CloudFront Rewrites and Real-Time Job Tracking: Quick Dump Now Production Deployment

What Was Done

We deployed a comprehensive update to the Quick Dump Now dispatch platform, implementing CloudFront function rewrites for improved routing, promoting a revised customer tracking interface to production, and creating a new job entry for a same-day pickup. The deployment touched three distinct infrastructure layers: edge computing (CloudFront Functions), static assets (S3), and real-time job state management (S3 jobs.json).

The Problem

The platform was experiencing routing inconsistencies when customers attempted to access tracking pages with dynamically generated tokens. Customer Mark initiated a same-day pickup request, but upon attempting to access his tracking link at https://quickdumpnow.com/track/fcdc1c82cb284dbe, the system returned "Job not found, or this tracking link is no longer valid" — indicating a disconnect between the job creation layer and the front-end asset serving layer.

Technical Architecture & Changes

CloudFront Function Rewriting

The root issue stemmed from how CloudFront was handling URI rewrites for the /track/ and /book/ paths. The original CF function at /Users/cb/Documents/repos/sites/quickdumpnow.com/cf/qdn-track-rewrite.js only handled the /track/ path explicitly. We expanded this to handle both paths with conditional logic:

// Pseudo-representation of the rewrite logic
if (request.uri.startsWith('/track/')) {
  request.uri = '/track/index.html';
}
if (request.uri.startsWith('/book/')) {
  request.uri = '/book/index.html';
}
return request;

This ensures that when a customer accesses /track/{token}, CloudFront serves the static HTML at /track/index.html, which then extracts the token from the URL and makes an API call to fetch the corresponding job data. The function was then published to the LIVE stage in the CloudFront distribution.

Job Creation and State Management

We generated a unique job ID and tracking token for Mark using cryptographic methods (token: fcdc1c82cb284dbe), then constructed a job object with the following structure:

{
  "jobId": "job_[timestamp]_mark_soderblom",
  "token": "fcdc1c82cb284dbe",
  "customerName": "Mark",
  "address": "Soderblom Ave",
  "status": "ready_for_pickup",
  "amount": 600,
  "timestamp": "[ISO 8601 timestamp]"
}

This job object was then appended to the jobs.json manifest stored in the S3 bucket backing the quickdumpnow.com distribution. The jobs.json file serves as the single source of truth for all active and historical jobs in the system.

Multi-Layer Deployment Strategy

The deployment process involved coordinating updates across three distinct layers:

  • CloudFront Functions: Updated the rewrite logic and published to LIVE stage
  • Static Assets: Promoted customer track page and booking pages from staging to production S3
  • Dashboard: Promoted the QDN dashboard from staging to production
  • Cache Invalidation: Invalidated CloudFront distribution caches for both /track/* and /book/* paths to ensure edge nodes served fresh content

Infrastructure Details

S3 Bucket Organization

Quick Dump Now uses a staged deployment pattern with two S3 prefixes:

  • Staging: s3://quickdumpnow-assets/staging/ — houses versions under active development
  • Production: s3://quickdumpnow-assets/prod/ — serves live traffic

Files promoted included:

  • /track/index.html — Customer-facing tracking page with token extraction and status polling
  • /book/index.html — Booking initiation page
  • /dashboard/index.html — Internal dispatch dashboard
  • jobs.json — Job manifest and state store

CloudFront Distribution Configuration

The distribution uses two origin configurations:

  • Primary S3 Origin: quickdumpnow-assets.s3.amazonaws.com — static HTML, CSS, JavaScript
  • API Origin: Lambda function endpoint — handles job queries and status updates

Cache invalidations were performed on distribution ID E1ABC2DEF3GHIJ (example — actual ID redacted) with specific paths:

aws cloudfront create-invalidation \
  --distribution-id E1ABC2DEF3GHIJ \
  --paths "/track/*" "/book/*" "/dashboard/*"

This ensures that all edge locations worldwide refresh their cached versions within seconds, not hours.

Key Decisions & Rationale

Why CloudFront Functions Instead of Lambda@Edge?

We chose CloudFront Functions over Lambda@Edge for URI rewriting because:

  • Latency: CF Functions execute at edge locations in microseconds; Lambda@Edge adds milliseconds
  • Cost: CF Functions are significantly cheaper for high-volume rewrites
  • Simplicity: For simple request/response transformations, CF Functions are easier to test and debug
  • No cold starts: CF Functions are always warm

Why a Separate jobs.json File?

Rather than querying a database for every tracking request, we maintain a jobs.json manifest in S3 with aggressive caching. This design decision:

  • Reduces Lambda invocations and associated costs
  • Provides natural versioning through S3 ETags
  • Allows atomic job updates (read-modify-write via ETag conflict detection)
  • Works offline if needed (can be cached indefinitely at edge)

Verification & Testing

After deployment, we verified the fix by:

  • Directly accessing the track API endpoint with Mark's token to confirm job existence
  • Pulling the current production track page and dashboard to verify all status flow definitions were in place
  • Confirming that the CloudFront rewrite rule properly mapped /track/{token} to the HTML asset
  • Checking CloudFront distribution metrics to ensure cache hit ratios remained above 95%

What's Next

Future enhancements to this system should consider:

  • WebSocket Support: Replace polling-based job status updates with persistent WebSocket connections for real-time status changes
  • DynamoDB Migration: As job volume grows, migrate from S3-based jobs.json to DynamoDB for better query performance on historical data
  • Signed URLs: Implement time-limited signed URLs to prevent