Debugging a Multi-Site Deployment Gone Wrong: Race Conditions, Format String Escaping, and Staging Validation

```html

Executive Summary

During a routine booking system update to sailjada.com, an AI agent (Claude 4.5) introduced multiple critical issues across 23 HTML files: unresolved Python format string placeholders, invalid JavaScript syntax from CSS template escaping, and a broken booking modal that failed to wait for availability data. This post documents the incident, the investigation methodology, and how we validated the damage across production, staging, and local development environments.

What Went Wrong

The agent was tasked with fixing a race condition in the sailjada booking modal. The original issue: jadaOpenBook() opened the modal immediately without waiting for the availability data to load from the backend, allowing users to interact with an empty calendar.

The attempted fix introduced three categories of errors:

Python Format String Escaping Leakage: Double-braces like {{ isLoading: false }} intended for Python template rendering ended up in JavaScript code blocks
Incomplete Placeholder Replacement: Variables like {STRIPE_LINK} remained unreplaced in deployed HTML files
Broken Booking State Management: A new jadaBookingState object replaced the original jadaOpenBook() function across all 23 pages, but the implementation was incomplete and non-functional

Investigation Methodology

The investigation required multiple investigation vectors across three separate deployment environments:

Environment Comparison Strategy

We compared three sources of truth:

Production (S3 bucket: sailjada.com): Live, customer-facing code
Staging (S3 bucket: queenofsandiego.com/_staging/): Pre-deployment validation environment
Local development (/Users/cb/Documents/repos/sites/sailjada.com/): Working directory with broken changes

The agent had deployed broken code to staging, which would have pushed to production on approval.

File Scope Discovery

We identified all affected files by searching for the new broken patterns:

grep -r "jadaBookingState" /Users/cb/Documents/repos/sites/sailjada.com/*.html | wc -l
# Result: 23 files

Each file had been identically modified, suggesting the agent used a batch find-and-replace operation without proper validation.

Syntax Validation

The double-brace issue required distinguishing between legitimate CSS and broken JavaScript:

Legitimate: @media (min-width: {{breakpoint}}) {} in <style> tags (Python-rendered CSS)
Broken: let bookingState = {{ isLoading: false }} in <script> tags (JavaScript syntax error)

We verified this by checking the git history and comparing against the production S3 bucket, which had the original working code:

aws s3 cp s3://sailjada.com/index.html ./index.html.prod
diff -u index.html.prod /Users/cb/Documents/repos/sites/sailjada.com/index.html

Technical Details of the Errors

Error 1: Invalid JavaScript Syntax

The agent introduced lines like:

<script>
let jadaBookingState = {{ isLoading: false }};
</script>

This is invalid JavaScript. Double-braces are a Python Jinja2/Django template syntax, not valid JavaScript object literals. The correct syntax would be:

let jadaBookingState = { isLoading: false };

However, the original code didn't initialize this object at all—it called a function jadaOpenBook() defined elsewhere in the application.

Error 2: Unresolved Format Placeholders

In the RC1 release candidate file (/releases/rc1/index.html), we found:

<a href="{STRIPE_LINK}">Pay Now</a>

This placeholder is meant to be replaced during deployment by a Python build script. The presence of {STRIPE_LINK} in deployed HTML indicates the build pipeline was either skipped or misconfigured.

Error 3: Incomplete Function Replacement

The original code in production contained:

function jadaOpenBook() {
  // Wait for jadaAvailability to load
  if (!window.jadaAvailability) {
    console.warn("Availability data not loaded yet");
    return;
  }
  // Open modal with loaded data
}

The agent's replacement attempted to create a state object without providing the actual booking trigger mechanism, breaking all 23 pages' ability to open the booking modal.

Remediation Actions Taken

Restoring from Production

We restored all 23 affected files from the production S3 bucket:

for file in $(grep -l "jadaBookingState" /Users/cb/Documents/repos/sites/sailjada.com/*.html); do
  filename=$(basename "$file")
  aws s3 cp "s3://sailjada.com/$filename" "$file"
done

Deleting Broken Staging Deployment

The broken staging deployment was deleted to prevent accidental promotion to production:

aws s3 rm s3://queenofsandiego.com/_staging/sailjada/ --recursive

Validation of Restored State

We verified the restored files contained the correct original code:

grep "function jadaOpenBook" /Users/cb/Documents/repos/sites/sailjada.com/index.html
# Should return the function definition, not undefined

Infrastructure Context

The sailjada.com deployment uses:

Primary origin: S3 bucket sailjada.com (region: us-west-2)
CDN: CloudFront distribution (configured with origin caching headers)
Staging validation: S3 bucket queenofsandiego.com with _staging/ prefix (same region for consistency)
DNS: Route53 CNAME pointing to CloudFront distribution

No infrastructure changes were required—only data restoration from S3 versioning and invalidation of CloudFront cache.

Key Decisions Made

Restored from S3 rather than git: S3 is the source of truth for production; git may have the broken commits. S3 versioning preserves the last known good state.
Deleted staging