Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

```html

Diagnosing and Staging a Google Apps Script Deployment Outage: Access Control & Multi-Environment Management

What Happened

On 2026-06-02, the deposit/reservation widget across 11 public event pages went dark. All instances of the "Reserve" button — the entry point for inbound bookings and deposits — were returning HTTP 403 (access denied) or 404 (not found) responses. The root cause: a Google Apps Script project had lost its public deployment access, silencing the entire booking funnel without alerting the user.

This post documents the diagnosis process, the infrastructure decisions that led to the outage, and the staged fix that restores the revenue pipeline.

Technical Details: The Outage

The system uses Google Apps Script as a lightweight backend for the reservation widget. Two separate deployments existed:

  • Primary endpoint (10 pages): https://script.google.com/macros/s/{DEPLOYMENT_ID_1}/exec — returning 403
  • Secondary endpoint (worship page): https://script.google.com/macros/s/{DEPLOYMENT_ID_2}/exec — returning 404

HTTP 403 indicates the deployment exists but access control has been restricted (likely from "Anyone" → "Me" or a specific user/domain). HTTP 404 indicates the deployment was deleted entirely.

Discovery Process

The diagnosis followed this sequence:


# Step 1: Verify live endpoint status
curl -I https://script.google.com/macros/s/{DEPLOYMENT_ID_1}/exec
# Result: HTTP/1.1 403 Forbidden

curl -I https://script.google.com/macros/s/{DEPLOYMENT_ID_2}/exec
# Result: HTTP/1.1 404 Not Found

# Step 2: Locate the Apps Script project ID in source control
grep -r "scriptId" ~/projects/*/clasp.json
# Found in: ~/projects/sailjada-reserve/clasp.json

# Step 3: Map deployment IDs to project metadata
cat ~/projects/sailjada-reserve/appsscript.json
# Confirms: Project name "JADA Reserve Widget", ID "1dDpSK8JZda7...lQJ5"

# Step 4: Cross-reference jada-ops documentation
grep -r "Apps Script" ~/Library/Mobile\ Documents/com~apple~CloudDocs/jada-ops/
# Located staging notes in DEPOSIT-OUTAGE-FIX-2026-06-02.md

Root Cause Analysis

Two separate issues:

  • Deployment 1 (403): Access control was manually restricted. This typically happens when a deployment is edited in the Google Apps Script console and the "Who has access" setting is changed from "Anyone" to "Me" or a specific email/domain, then deployed without reverting the setting.
  • Deployment 2 (404): The deployment was manually deleted from the "Manage deployments" panel in the Google Apps Script console. This is a one-way action; the `/exec` URL becomes permanently invalid.

Why this happened: The most likely scenario is a developer testing or troubleshooting access control in the Google console accidentally saved a restrictive deployment or deleted a test deployment without realizing it was the one referenced by production pages.

Infrastructure: Google Apps Script Deployment Architecture

Google Apps Script uses a two-tier deployment model:

  • Project: The container for source code, version history, and metadata. Identified by a long alphanumeric ID.
  • Deployment: A snapshot of the project's code at a point in time, with its own access control settings and `/exec` URL. Multiple deployments can coexist in one project.

Key file locations in the source repo:

  • ~/projects/sailjada-reserve/clasp.json — Contains the Apps Script project ID and source root.
  • ~/projects/sailjada-reserve/appsscript.json — Project metadata, dependencies, and display name.
  • ~/projects/sailjada-reserve/src/ — TypeScript/JavaScript source files that get transpiled and deployed.

Deployments are immutable after creation; to update code, a new deployment must be created. The `/exec` URL remains valid as long as the deployment exists and access control is set to "Anyone" (for public-facing endpoints).

Staged Fix

The fix is a one-step manual action in the Google Apps Script console (no code changes required):


1. Navigate to script.google.com
2. Open project "1dDpSK8JZda7UXpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5"
3. Click "Deploy" → "Manage deployments"
4. For the active deployment:
   - Set "Who has access" = "Anyone"
   - Set "Execute as" = "Me" (the service account running the Apps Script)
5. Click "Deploy"

Why this works: Redeploying with "Anyone" access restores the public `/exec` URL to a callable state. The "Execute as" setting ensures the Apps Script runs under the service account identity, which has permission to write to the underlying Google Sheets used for reservations.

For the 404 endpoint: If the deployment cannot be recovered (it was deleted), a new deployment must be created from the same project source code. The `/exec` URL will change, requiring an update to all 1 page(s) that reference it.

Key Decisions & Rationale

  • Why not use a CI/CD pipeline for deployments? Current setup uses manual clasp push and clasp deploy commands. A proper solution would integrate deployment into the build pipeline (GitHub Actions → clasp → Apps Script), with version pinning and automated rollback capability. This would prevent accidental deletions and enforce access control policies.
  • Why multiple deployments? Separate deployments for different sets of pages allows for independent access control and code versions. However, this introduces operational complexity; consolidating to a single deployment per project would reduce failure surface.
  • Monitoring gap: No alerting currently exists for 403/404 responses from the Apps Script endpoints. Adding HTTP health checks to a monitoring system (CloudWatch, Datadog, etc.) with Slack notifications would catch this outage in real-time.

What's Next

  • Immediate (post-fix): After the Google console redeploy, verify both endpoints return 200 with valid JSON responses. Test the reserve widget on all 11 pages.
  • Short-term: Document the Apps Script project ID, deployment IDs, and associated pages in a single source-of-truth (e.g., jada-ops/APPS-SCRIPT-INVENTORY.md).
  • Medium-term: Automate deployments via GitHub Actions. Move clasp deploy into the CI/CD pipeline with version tags and rollback procedures.
  • Long-term: Add synthetic monitoring for all Apps Script endpoints. Set up alerts in Slack for 4xx/5xx responses or elevated latency.