Preventing S3 Deployment Regressions: A Hard-Rules Framework for Static Site CI/CD
Last week, a deployment to our Queen of San Diego site wiped three working features by pushing a stale local index.html over a newer production version in S3. The hero crossfade animation (JADA → BOOK NOW), the Stripe embedded checkout flow, and a deliberately-removed hero line all vanished. This post documents the failure mode, the hard rules we built to prevent it, and how to apply them to similar static-site workflows.
What Happened: The Stale-Local Deployment Pattern
- Root cause: Local development environment had an older snapshot of
/index.html(last edited 48 hours prior). Production S3 had a newer version (edits from yesterday). A deploy command copied the stale local file to S3 without checking what was already there. - Impact: Three independent features regressed:
- JADA booking hero crossfade (CSS animation + JavaScript state)
- Stripe embedded checkout integration (removed from older local version)
- "For Ranch & Coast readers…" hero text (had been deleted in prod, resurrected by stale local)
- Detection lag: ~2 hours before visual inspection caught it. No automated diff or diff-before-deploy step existed.
The Hard-Rules Framework: Eight Preventative Controls
We codified eight rules directly into /Users/cb/Documents/repos/sites/queenofsandiego.com/CLAUDE.md to load automatically on every session:
D1: Pull S3 and Diff Before Any Edit
aws s3 cp s3://queenofsandiego-prod/ ./s3-prod-snapshot/ --recursive
diff -r ./s3-prod-snapshot/ ./src/ > deployment-diff.txt
# Review diff.txt before making local changes
Why: Establishes ground truth. You can't regress what you've explicitly seen.
D2: Staging-Only, Single-Target Deploys
Never deploy staging and prod in the same command. Staging first, review, then promote to prod.
# Deploy to staging only
aws s3 sync ./src/ s3://queenofsandiego-staging/ --delete
# After review and approval:
aws s3 sync ./src/ s3://queenofsandiego-prod/ --delete
Why: Isolates blast radius. A broken deploy hits staging; prod waits for human eyeballs.
D3: One File Per Logical Change
Don't batch unrelated edits (hero animation + pricing math + footer copy) into a single deploy. Deploy, verify, then deploy next change.
Why: Pinpoints regressions to their commit. "Pricing math broke the hero fade" is obvious; "three things in one deploy broke one thing" is a debugging nightmare.
D4: Obey Prior Session-Summary Warnings
If a prior session's summary explicitly warns "local files are stale," don't proceed without re-syncing from S3 first (D1).
Why: Institutional memory. Your own prior work already spotted the risk; ignoring it guarantees failure.
D5: Snapshot Production Before Overwriting
aws s3 sync s3://queenofsandiego-prod/ ./snapshots/prod-$(date +%Y%m%d-%H%M%S)/ --recursive
# Now deploy to staging, verify, then to prod
Why: S3 versioning is disabled on our buckets (for cost). Manual snapshots are your rollback.
D6: Proof Block Before Any cp or sync
Before running a deployment command, print six lines to chat:
- Source file/directory being deployed
- Target S3 bucket and prefix
- Last modification time of source
- Last modification time of current prod object(s)
- Diff of key sections (hero animation, checkout form, etc.)
- Explicit statement: "Deploying to staging only" or "Promoting staging to prod after review"
Why: Forces deliberate thought. Printing stale timestamps is embarrassing; it stops you.
D7: Feature-Token Registry
Maintain a FEATURES.md in the site root with grep-friendly markers:
Before deploying, grep S3-current against your local file:
aws s3 cp s3://queenofsandiego-prod/index.html ./s3-current.html
grep "FEATURE:" ./src/index.html > local-features.txt
grep "FEATURE:" ./s3-current.html > prod-features.txt
diff local-features.txt prod-features.txt
Why: Detects silent regressions. If STRIPE_EMBEDDED_CHECKOUT vanishes from the diff, deployment is blocked until investigated.
D8: Escalate to CB If S3 Is Ahead of Local
If step D1 reveals that prod has features not in your local working directory, stop. Message CB before proceeding.
Why: Indicates either a prior session edited prod directly (bad practice), or you're on a stale branch. Either way, your next deploy needs human context.
Infrastructure & CloudFront Invalidation
QOS uses three S3 buckets:
queenofsandiego-staging— CloudFront distribution IDE1ABCD1234EFGHqueenofsandiego-prod— CloudFront distribution IDE5XYZ9876IJKLqueenofsandiego-backup— manual snapshots only, no CDN
After any S3 sync, invalidate CloudFront cache:
aws cloudfront create-invalidation --distribution-id E1ABCD1234EFGH --paths "/*"
aws cloudfront create-invalidation --distribution-id E5XYZ9876IJKL --paths "/*"
Session-Memory Automation
The rules are baked into CLAUDE.md at the site root. On every new session, the rules auto-load as context. Additionally, a condensed reference pointer was added to the top-level /Users/cb/Documents/repos/CLAUDE.md so other static sites benefit from the pattern without duplication.