```html

Diagnosing and Staging a Deposit-Collection Outage: Apps Script Access Control and Multi-Region Deployment Recovery

This post covers the incident response workflow for a production deposit-collection outage affecting eleven event-booking pages across sailjada.com and queenofsandiego.com. The core issue: two Google Apps Script deployments returning 403 (Forbidden) and 404 (Not Found) responses, silently breaking the "Reserve" widget on every public event page. We'll walk through the diagnostic pipeline, infrastructure state verification, and the staged fix ready for immediate deployment.

The Problem: Dark Funnel, Silent Failures

The deposit collection system relies on three Google Apps Script endpoints:

• Primary endpoint (10 pages): `https://script.google.com/macros/s/AKfycbz...44Pme8wCA/exec` — Returns 403 Forbidden
• Worship/secondary endpoint (1 page): `https://script.google.com/macros/s/AKfycbz...AFsLWaO3/exec` — Returns 404 Not Found
• Fallback/tertiary endpoint (staged for recovery): Project ID `1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5`

Unlike a visible HTTP 500 error, a 403/404 on an async endpoint silently fails at the client level. Users see an unresponsive button, no error feedback. The booking funnel darkens—we can't measure lost deposits because the telemetry never fires. This is a revenue-stopping condition.

Diagnostic Workflow: From Project ID to Live Endpoint State

The first challenge: the Apps Script ecosystem conflates three identifiers—project ID, deployment ID, and executable URL—making it easy to verify the wrong thing.

Step 1: Verify Project Accessibility and Locate Deployment Config

We confirmed the project exists and is readable at script.google.com, then searched for its deployment metadata:

# Locate clasp config files to find project associations
find ~/Documents/repos -name ".clasp.json" -exec grep -l "1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5" {} \;

# Read project metadata from appsscript.json
cat ~/Documents/repos/jada-deposit-proxy/appsscript.json | grep -A5 "displayName"

The project is named "JADA Deposit Collection (Primary)" and is tracked in source control. The clasp config confirms its live deployment slot.

Step 2: Test Live Endpoint Responses (Before Fix)

Before any changes, we captured the exact failure state:

# Test primary endpoint
curl -i https://script.google.com/macros/s/AKfycbz...44Pme8wCA/exec \
  -H "Content-Type: application/json" \
  -d '{"action":"testConnection"}' 2>&1 | head -20

# Returns: HTTP/1.1 403 Forbidden
# Body: "Error: Access Denied. You do not have permission to access this resource."

# Test worship/secondary endpoint  
curl -i https://script.google.com/macros/s/AKfycbz...AFsLWaO3/exec 2>&1 | head -20

# Returns: HTTP/1.1 404 Not Found

The 403 indicates the deployment exists but access control is misconfigured (likely "Owner only" or a specific user list that's expired). The 404 on worship suggests that deployment was deleted or never redeployed after a migration.

Step 3: Verify Event Pages Are Still Calling Old Endpoints

We then dry-ran a full endpoint rewrite across all eleven event pages to confirm they're still bound to the broken URLs:

# Search event page HTML in S3 for old endpoint references
aws s3 cp s3://queenofsandiego-assets/events/ . --recursive --exclude "*" \
  --include "*.html" | xargs grep -l "44Pme8wCA/exec"

# Returns: 10 pages still calling the 403 endpoint
# Dry-run rewrite (no --apply flag, just log what would change)
./tools/rewrite_endpoints.sh --source s3://queenofsandiego-assets/events/ \
  --old-endpoint "44Pme8wCA/exec" \
  --new-endpoint "JADA-Deposit-Primary-v2/exec" \
  --dry-run

This confirmed the funnel is indeed dark and ready to be fixed.

Infrastructure: Google Apps Script Deployment Architecture

Google Apps Script deployments follow a specific naming and access model that's easy to misconfigure:

• Project ID — immutable, used by clasp for source control only; not called by end users
• Deployment ID — created when you hit "Deploy → New Deployment"; tied to a specific code version
• Exec URL — `script.google.com/macros/s/{deploymentId}/exec`; this is what the client calls
• Access Control — set per deployment: "Only myself," "Specific people," or "Anyone"
• Execute As — which identity runs the Apps Script code: "Me" (script owner) or the calling user

The gotcha: redeploying into the *same* deployment slot keeps the URL unchanged, but redeploying into a *new* slot generates a new URL and breaks all callers unless they update their links. Our strategy: redeploy into the same slot (no client changes needed) and flip the access to "Anyone" (since the endpoint is called from public web pages).

The Fix: Access Control Redeployment

The single action required is a re-share in the Google Apps Script console:

1. Navigate to script.google.com and open project 1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5
2. Click Deploy → Manage Deployments
3. Select the current "Head deployment" (or create a new one if none exist)
4. Set Who has access: Anyone
5. Set Execute as: Me (to run under the script owner's authorization)
6. Click Deploy

This redeploys into the existing slot, keeping the `/exec` URL identical. The 403 becomes 200, and the funnel reopens within seconds (no CloudFront cache invalidation needed—Apps Script bypasses CDN).

Staging for the Worship Page: Recovery Path

The worship page's 404 indicates its deployment was deleted. Two paths: