Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

```html

Building a Real-Time GA4 Analytics Pipeline with Orchestrator Integration and Deep-Link Dashboard Navigation

Over the past development session, we built out a comprehensive Google Analytics 4 (GA4) data pipeline that connects directly to our orchestrator infrastructure, enabling automated reporting, traffic analysis, and operational insights. This post walks through the technical architecture, authentication patterns, and the deep-linking dashboard system that surfaced critical findings to stakeholders in real time.

What Was Done

We accomplished three core objectives:

  • Established programmatic GA4 Data API access with proper OAuth 2.0 service account authentication
  • Pulled 30-day historical traffic data across all Queen of San Diego properties and fed it into the orchestrator for analysis
  • Built a hash-navigation deep-linking system on the progress dashboard to surface audit findings as actionable kanban cards

The result was a live audit card (t-31aa2593) on the progress dashboard containing five sections of findings: GA code gaps by site, traffic trend analysis, email campaign status, and operational excellence recommendations.

Technical Architecture: GA4 Authentication and Data Retrieval

The core challenge was establishing headless access to GA4 data without browser-based OAuth flows. We implemented a service account pattern using the Google Analytics Data API v1.

Authentication Flow:


# File: /Users/cb/Documents/repos/tools/reauth_ga.py

import google.auth.transport.requests
from google.oauth2.service_account import Credentials
from google.analytics.data_v1beta import BetaAnalyticsDataClient
from google.type.date_range_pb2 import DateRange

# Load service account credentials from JSON keyfile
credentials = Credentials.from_service_account_file(
    keyfile_path='path/to/service-account-key.json',
    scopes=['https://www.googleapis.com/auth/analytics.readonly']
)

# Initialize authenticated client
client = BetaAnalyticsDataClient(credentials=credentials)

# Query GA4 property for last 30 days of traffic
property_id = "properties/XXXXX"  # From GA Admin console
date_range = DateRange(start_date="30daysAgo", end_date="today")

request = RunReportRequest(
    property=property_id,
    date_ranges=[date_range],
    dimensions=[Dimension(name="pagePath"), Dimension(name="deviceCategory")],
    metrics=[Metric(name="activeUsers"), Metric(name="screenPageViews")]
)

response = client.run_report(request)

Why This Approach: The GA4 Data API requires a service account because we're running automated, scheduled jobs that don't have a user to perform interactive OAuth. Service accounts use a JSON keyfile containing a private key that the client library uses to sign JWT requests directly to Google's token endpoint. This eliminates the need for browser redirects or manual token refresh cycles.

The property ID came from the screenshot URL in the dashboard settings—a pattern we documented in the memory system for future audits. The read-only scope constraint follows the principle of least privilege: the service account can only query GA data, not modify properties or other accounts.

GA Code Audit Across All Properties

Before we could trust the traffic data, we needed to verify that GA measurement code was actually present on every page across all platforms. We built a file-system audit that swept across the codebase:

  • Scanned all HTML templates in `/Users/cb/Documents/repos/*/templates/` and `*/public/` directories
  • Looked for GA4 gtag scripts with the measurement ID pattern `G-XXXXXXXXXX`
  • Logged files missing measurement code and flagged them for remediation
  • Generated a report broken down by site/platform

The audit discovered gaps where certain pages (particularly older checkout flows and legacy admin dashboards) lacked GA code entirely. These were surfaced as sub-items in the dashboard card so developers could prioritize fixes by traffic impact.

Dashboard Deep-Linking and Hash Navigation

The progress dashboard at https://progress.queenofsandiego.com uses client-side hash routing to enable deep links directly to specific cards. This was critical for making the audit findings accessible without burying them in console output.

Deep Link Format:


# Card reference pattern
https://progress.queenofsandiego.com/#card-t-31aa2593

# The hash fragment is parsed by the dashboard JS router:
window.location.hash = '#card-t-31aa2593'

The dashboard JavaScript (in the SPA's router module) listens for hash change events and renders the appropriate card by ID. When the audit completed, the orchestrator logged the card ID, and we immediately provided stakeholders with the direct link. No navigation required—users land directly on the findings.

Card Structure: Each audit finding is a kanban card with five sections:

  1. GA code coverage by site (missing measurement IDs highlighted)
  2. 30-day traffic trends (active users, page views, bounce rate)
  3. Traffic recommendations (low-traffic pages to optimize, high-bounce-rate flows to investigate)
  4. Email campaign audit (scheduled Constant Contact blasts with approval status)
  5. Operational excellence gaps (deployment frequency, error rates, response times)

Orchestrator Integration

The orchestrator process received a full brief containing:

  • Raw GA4 API response (JSON) with all traffic metrics for the past 30 days
  • List of all HTML files scanned and their GA code status
  • Current Constant Contact campaign list with send dates and approval status
  • Prompt asking for specific recommendations on traffic growth and email operational excellence

The orchestrator synthesized this data into the five-section card format and assigned it a unique ID. The card was written to the dashboard's data store (backing this could be DynamoDB, PostgreSQL, or a flat JSON file depending on deployment) and made immediately queryable via the card ID.

Key Decisions and Rationale

Why GA4 Data API vs. GA4 Reporting API: The GA4 Data API is the newer, preferred endpoint and includes support for more granular dimensions and filters. The older Reporting API was deprecated for new integrations.

Why Service Account Auth: Eliminates token refresh management. A service account's credentials are long-lived (the private key in the JSON file doesn't expire). This is essential for scheduled batch jobs that run without human intervention.

Why Deep-Link Hashes Instead of Query Params: Hash fragments don't trigger server-side routing, so the dashboard SPA can handle navigation entirely on the client. This is critical when the dashboard is deployed behind a CDN (CloudFront) with aggressive caching—hash changes never hit the origin.

Why File-System Audit Before API Queries: We needed to know which pages should be tracked before trusting traffic numbers. If a page is missing GA code, zero traffic doesn't mean it's not being visited—it just means we're not measuring it.

What's Next

The audit surfaced three urgent items requiring immediate action:

  1. Mother's Day Email Campaign: Scheduled for April 29, still in needs-approval status. Review card t-XXXXX on the dashboard.
  2. Paul Simon Blast Proof: Due May 12. Needs legal review before send.