Preventing S3 Deployment Regressions: Hard Rules for Multi-Environment CloudFront Sites

```html

Over a recent three-hour session, a deployment incident wiped three working features on a production CloudFront site by pushing a stale local index.html over a newer S3 version. The JADA→BOOK NOW hero crossfade, Stripe embedded checkout flow, and previously-removed Ranch & Coast copy all vanished. The root cause: skipping a pre-deploy diff check and violating the staging-first rule in a single command. Here's how we hardened the workflow to prevent it.

What Happened

The deployment command looked innocent:

aws s3 cp index.html s3://queenofsandiego.com/index.html
aws s3 cp index.html s3://queenofsandiego.com-staging/index.html

Two problems:

Stale local file: The developer's local /Users/cb/Documents/repos/sites/queenofsandiego.com/index.html was 36 hours old; S3 prod had been updated in the last session. No pre-deploy pull or diff.
Simultaneous dual-deploy: Both prod and staging got overwritten in the same session, violating the critical pattern: stage first, review, then promote.

Result: Three features regressed. No snapshot. No way to revert safely without rebuilding from git history (which didn't contain the S3-only changes).

The Hard Rules (D1–D8)

We codified eight non-negotiable deployment rules into the project's CLAUDE.md files so they auto-load on every session:

D1: Pull and Diff Before Any Edit

Before touching a file destined for S3, always fetch the live version and diff it locally:

aws s3 cp s3://queenofsandiego.com/index.html ./index.html.prod
diff -u index.html.prod index.html

Why: Catches stale local files immediately. Non-negotiable on any S3 bucket used by a live site.

D2: Staging-Only Single-Target Deploys

Each deploy command targets exactly one environment. Always stage first:

# CORRECT
aws s3 cp index.html s3://queenofsandiego.com-staging/index.html
# Later, after review:
aws s3 cp index.html s3://queenofsandiego.com/index.html

# WRONG
aws s3 cp index.html s3://queenofsandiego.com/index.html s3://queenofsandiego.com-staging/index.html

Why: One target per command enforces deliberate staging review. Batch commands hide mistakes and prevent rollback.

D3: One Logical Change Per Deploy

If you're shipping the booking flow, Stripe keys, and email template in one session, deploy them separately:

Commit 1: Booking flow HTML + logic (stage, review, promote).
Commit 2: Stripe Session changes (stage, review, promote).
Commit 3: Email template.

Why: If one breaks, you know exactly what to revert. Easier to diagnose which feature caused a regression.

D4: Obey Your Own Prior Session Warnings

Every session ends with a summary. If your previous summary said "local index.html is 36 hours stale," and the next session starts on the same file, pull first. No exceptions.

Why: Sessions are state machines. Violations are bugs.

D5: Snapshot Prod Before Overwrite (No S3 Versioning Fallback)

S3 versioning is disabled on queenofsandiego.com to avoid cost and clutter. Before any cp to prod, snapshot:

aws s3 cp s3://queenofsandiego.com/index.html ./snapshots/index.html.$(date +%s)
# Now safe to deploy
aws s3 cp index.html s3://queenofsandiego.com/index.html

Why: Manual snapshots are cheap, fast, and give you a recovery point even without versioning. Commit them to git.

D6: Six-Line Proof Block Before Any `cp`

Before deploying, print this in chat:

DEPLOY PROOF:
Source: /path/to/local/file (modified 2025-01-15T14:32:10Z)
Target: s3://bucket/path (current prod: 2025-01-14T08:22:00Z)
Diff: [actual diff output, first 10 lines]
Staging deployed: YES/NO (URL: staging.example.com)
Approval: [from CB or your own explicit sign-off]

Why: Forces you to state facts out loud. Catches lies (stale file claims, missing staging tests, etc.) before they ship.

D7: Feature Token Registry

Keep a FEATURES.md in the repo listing every live feature and the files it touches:

# queenofsandiego.com Features

## JADA → BOOK NOW Hero Crossfade
- **Files:** index.html (line 847–892), styles.css (line 341–356)
- **S3 Deployed:** 2025-01-14T08:22:00Z
- **Tokens:** <div id="jada-hero">, .hero-fade-in, data-next="BOOK NOW"
- **Last Verified:** 2025-01-14

## Stripe Embedded Checkout
- **Files:** index.html (line 2104–2156), /js/stripe-session.js (git-tracked)
- **S3 Deployed:** 2025-01-13T16:45:00Z
- **Tokens:** Stripe.redirectToCheckout(), sessionId: window.STRIPE_SESSION
- **Last Verified:** 2025-01-13

Before and after deploy, grep S3 current HTML for the tokens:

aws s3 cp s3://queenofsandiego.com/index.html - | grep -c "jada-hero"
# Should output: 1 (or 2, depending on template). Regression = 0.

Why: Automated smoke tests. Catches silent regressions (missing HTML sections) immediately.

D8: Escalate to CB if S3 is Ahead of Local

If the diff shows S3 has content not in your local file, stop and escalate:

aws s3 cp s3://queenofsandiego.com/index.html ./index.html.prod
diff -u index.html index.html.prod | grep "^>" | head -5
# If output is non-empty, you're about to revert something.
# Message CB with the diff. Do not deploy.

Why: Protects against accidental reversions of work done in prior sessions.

Infrastructure Details

← All posts