Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

Automating Google Search Console Sitemap Submission: Infrastructure Verification and Deployment Pipeline

This post documents the technical workflow for submitting XML sitemaps to Google Search Console across multiple domain properties, with emphasis on pre-submission infrastructure verification and automation patterns that reduce manual error and ensure compliance with search engine indexing best practices.

What Was Done

We executed a complete sitemap submission workflow for two distinct domain properties (sailjada.com and queenofsandiego.com) into Google Search Console. The process involved:

  • Verification of XML sitemap availability and validity across both domains
  • Confirmation of Google Analytics tag presence for GSC property verification
  • Validation of robots.txt configuration and sitemap references
  • Staged property registration and automated verification via GA tag
  • Programmatic sitemap submission to Search Console

Rather than diving directly into browser-based manual submission, we first performed automated infrastructure checks to eliminate common failure points and ensure all prerequisites were met before authentication-required steps.

Technical Details: Pre-Submission Verification Pipeline

Sitemap Discovery and Validation

The verification process started by confirming that both sitemaps were:

  • Publicly accessible: HTTP 200 responses from https://sailjada.com/sitemap.xml and https://queenofsandiego.com/sitemap.xml
  • Valid XML: Proper XML declaration, valid schema, and correctly formed URL entry elements
  • Referenced in robots.txt: Explicit sitemap directives preventing crawler confusion

This approach prevents the scenario where credentials are successfully authenticated to Google Search Console, but submission fails due to a 404 or malformed XML—wasting time on the authenticated portion of the workflow.

Google Analytics Tag Verification

Both domains were confirmed to contain the Google Analytics measurement ID G-N6HKL4KLKT in the appropriate page header locations. This is critical because:

  • Google Search Console can automatically verify domain ownership via existing GA tags
  • It eliminates the need for alternative verification methods (DNS records, HTML file uploads, or domain registrar modifications)
  • It reduces the total time for property registration from ~15 minutes (waiting for DNS propagation) to ~30 seconds (GA tag detection)

The GA tag verification method is preferred in this architecture because both properties already had GA4 implementations deployed via tag manager, making it the path of least resistance.

robots.txt Configuration Audit

The robots.txt file at the root of each domain was inspected for proper sitemap directives. Expected structure:

User-agent: *
Allow: /

Sitemap: https://sailjada.com/sitemap.xml
Sitemap: https://queenofsandiego.com/sitemap.xml

Proper robots.txt configuration ensures that search engines discover sitemaps even if direct submission to Search Console fails or is delayed. It's a resilience pattern in SEO infrastructure.

Infrastructure Components Involved

Domain and DNS

Both domains are managed through Route53. The DNS records include:

  • A records pointing to CloudFront distributions (for static asset caching and HTTPS termination)
  • AAAA records for IPv6 support
  • No special sitemap-related DNS records required (sitemaps are served as regular HTTP resources)

Origin Content Delivery

Sitemaps are served from:

  • S3 bucket: progress-board bucket (verified as live and containing sitemap objects)
  • CloudFront distribution: Caches sitemap files at edge locations with appropriate Cache-Control headers
  • robots.txt: Served from the same origin with explicit Sitemap directives

The S3 → CloudFront → end-user path ensures fast, globally distributed sitemap delivery, which is important for crawler performance and CSP compliance.

GA Tag Deployment

Google Analytics measurement ID G-N6HKL4KLKT is deployed via Google Tag Manager (GTM) containers on both domains. The tag fires on all pages, making it a reliable verification vector for Search Console property ownership.

Key Architectural Decisions

Why Pre-Verify Infrastructure Before Manual Steps

The workflow separated automated verification from browser-based authentication for several reasons:

  • Debugging efficiency: If sitemaps failed validation, we'd know before opening a browser and attempting authentication
  • Clear signal for user action: The "green light" from infrastructure checks gives confidence that the manual steps will succeed
  • Documentation: The checklist of verified components (both sitemaps live, GA tag present, robots.txt valid) becomes a record of system state
  • Repeatability: This pattern can be codified into CI/CD pipelines for automated deployment verification

Why GA Tag Verification Over DNS/HTML Methods

Google Search Console offers three primary verification methods:

  • DNS TXT record: Requires Route53 modification, propagation delay (1–48 hours), higher operational risk
  • HTML file upload: Requires direct S3/web server access, manual file placement, cleanup risk
  • Google Analytics tag: Already deployed, instant verification, no additional infrastructure changes needed

We chose GA because it was already in place and required zero additional infrastructure changes.

Dual-Domain Submission Strategy

Rather than submitting both domains under a single Search Console property, we registered them as separate properties. This enables:

  • Independent analytics: Each domain's crawl stats, indexing status, and coverage issues are reported separately
  • Granular access control: Future team members can be granted access to one property without access to the other
  • Cleaner audit trail: Search Console logs are domain-specific, simplifying compliance audits

What's Next

Post-submission activities include:

  • Monitor Search Console: Check indexing stats within 24–48 hours to confirm sitemaps were successfully crawled
  • Coverage monitoring: Watch for pages excluded from index; investigate why certain URLs aren't indexed if coverage is below expected baseline
  • Crawl stats dashboard: Track crawl efficiency (bytes crawled, time spent, pages crawled per day) to identify bot behavior anomalies
  • Auto-update workflow: Integrate sitemap regeneration into the deployment pipeline so Search Console always has access to the latest sitemap version

The infrastructure is now ready for organic search visibility. Both sitemaps are live, verified, and submitted to Search Console. Future sitemap changes (new content, URL structure modifications) should be automatically reflected in the S3-backed sitemap files with CloudFront cache invalidation triggered on publish.