Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

```html

Preventing Production Regressions: Hard Rules for Safe S3 Deployments

Last session, a well-intentioned deployment to queenofsandiego.com silently overwrote three working features on production—the hero JADA→BOOK NOW crossfade animation, the Stripe embedded checkout booking flow, and resurrected a previously-deleted "For Ranch & Coast readers..." hero line. Root cause: deploying a stale local index.html over a newer S3 production version, combined with simultaneous staging + prod deploys that violated prior session guardrails. This post documents the eight hard rules we've now encoded into the codebase to prevent this class of failure.

What Went Wrong

The incident followed this pattern:

  • Stale local state. Local /Users/cb/Documents/repos/sites/queenofsandiego.com/index.html was several commits behind S3 prod; git was up-to-date, but the working tree was not.
  • No pre-flight diff. The session made edits and immediately deployed without pulling S3 current and diffing against local.
  • Dual-target deploy. A single command pushed to both staging and prod in the same aws s3 cp, violating the "staging first, review, promote" workflow documented in prior session notes.
  • Ignored prior warnings. The previous session's summary explicitly flagged "stale local files—always pull S3 before edit," but this was not re-checked at session start.
  • No proof block. No human-readable cp command was printed in chat before execution, so regression was invisible until testing.

Impact: Three features went dark on production until the correct version could be restored from CloudFront cache headers and manual S3 recovery.

The Eight Hard Rules (D1–D8)

We've codified eight rules into /Users/cb/Documents/repos/sites/queenofsandiego.com/CLAUDE.md, which loads automatically at session start for any work on Queen of San Diego:

D1: Always pull S3 current before editing.

aws s3 cp s3://queenofsandiego.com/index.html ./index.html.s3current
diff -u index.html.s3current index.html

If S3 is ahead (newer production changes not in git), abort and escalate to CB. This prevents silent overwrites of live features.

D2: Staging-only, single-file deploys.

Never deploy to both staging and prod in one command. Deploy to s3://qos-staging/ first:

aws s3 cp index.html s3://qos-staging/index.html --cache-control "max-age=0"

Wait for CB to verify on staging.queenofsandiego.com, then—in a separate, explicit command—promote to prod:

aws s3 cp s3://qos-staging/index.html s3://queenofsandiego.com/index.html --cache-control "max-age=3600"

D3: One file per logical change.

Do not batch unrelated edits into a single file and deploy as one unit. If you're fixing the hero fade AND updating the booking button, make two separate files, test both on staging in sequence, and promote each once verified. This isolates regression surface.

D4: Obey your own prior session-summary warnings.

At session start, read the prior summary in CLAUDE.md. If it says "stale local" or "S3 ahead," re-check before touching anything. Treat prior-self as authoritative.

D5: Snapshot prod before any overwrite.

S3 versioning is not enabled on queenofsandiego.com (cost + complexity). Instead, before overwriting, always:

aws s3 cp s3://queenofsandiego.com/index.html ./backups/index.html.prod.$(date +%s)

Store locally with timestamp. If regression is detected, restore from this snapshot.

D6: Print a proof block before any cp.

Before running aws s3 cp, print the exact source file, exact target S3 path, and exact file size/hash:

Source: ./index.html
Target: s3://queenofsandiego.com/index.html
Size: 3650 lines, 487KB
MD5: abc123def456...
CloudFront dist: EXAMPLEABCD1234
Invalidation: /* (full cache clear)

This creates a human-readable audit trail in chat before the operation executes.

D7: Maintain a feature-token registry.

Critical features (hero fade, Stripe checkout, booking button) are tagged with unique tokens in the source:

<!-- FEATURE_TOKEN: JADA_BOOK_NOW_FADE_v2 -->
<!-- FEATURE_TOKEN: STRIPE_EMBEDDED_v3 -->
<!-- FEATURE_TOKEN: HERO_RANCH_COAST_REMOVED -->

After any deploy, grep S3 current for these tokens and confirm they exist and match expected versions:

aws s3 cp s3://queenofsandiego.com/index.html - | grep FEATURE_TOKEN

If expected tokens are missing, immediately roll back using D5 snapshot.

D8: Escalate to CB if S3 is ahead of local.

If D1 diff shows S3 has changes git doesn't know about, do not proceed. This is a sign of manual edits, prior-session carry-over, or out-of-band deployments. Message CB with the diff and wait for direction before touching anything.

Infrastructure Context

These rules sit atop the following architecture:

  • S3 buckets: queenofsandiego.com (production) and qos-staging (staging). Both are private; CloudFront serves them publicly.
  • CloudFront distribution: EXAMPLEABCD1234 (example ID; real ID in AWS Console). Origin is S3 bucket. TTL is 3600s prod, 0s staging.
  • Git repo: /Users/cb/Documents/repos/sites/queenofsandiego.com/. Tracked files include index.html, css/*, js/*. Local .gitignore excludes backups/ and S3 pull-downs like *.s3current.
  • Route53: DNS for queenofsandiego.com points CloudFront alias to distribution. staging.queenofsandiego.com points to staging distribution (separate dist ID).

Why These Rules Matter

S3 is stateless and unforgiving. Once you overwrite a file, the prior version is