Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

Resolving DNS Subdomain Shadowing: How a Wildcard CNAME Blocked adamcherrycomics.dangerouscentaur.com

The Problem

The subdomain http://adamcherrycomics.dangerouscentaur.com/ went down for several hours. The CloudFront distribution (E2Q4UU71SRNTMB) was healthy and returning 200 responses with valid content. DNS queries, however, were failing to resolve the subdomain entirely. This created a classic scenario where infrastructure was operational but unreachable due to DNS misconfiguration.

Root Cause: RFC 1034 DNS Shadowing

The root cause was a subtle but critical DNS behavior outlined in RFC 1034 § 4.3.3. The domain dangerouscentaur.com had two relevant CNAME records at Namecheap:

  • A wildcard CNAME: *.dangerouscentaur.com → CloudFront distribution
  • An explicit CNAME: www.adamcherrycomics.dangerouscentaur.com → CloudFront distribution

The problem: when the DNS resolver queried for adamcherrycomics.dangerouscentaur.com, it found that a DNS node explicitly exists for adamcherrycomics (created by the www.adamcherrycomics record). According to RFC 1034, once an explicit node exists at a level, wildcard matching does not apply to that level. The resolver looked for adamcherrycomics.dangerouscentaur.com, found a node but no matching record, and returned NXDOMAIN (non-existent domain).

Diagnostic Process

The troubleshooting sequence systematically eliminated possibilities:

  • CloudFront health: Direct curl to the distribution with explicit Host headers returned 200, confirming the origin was serving content correctly
  • DNS resolution: Queries against major public resolvers (Google, Cloudflare) and Namecheap's authoritative nameservers directly revealed no resolution
  • Authoritative nameserver queries: Querying pdns1.namecheap.com and pdns2.namecheap.com confirmed the issue was at the authoritative level, not a propagation delay

Commands used for verification:

curl -v -H "Host: adamcherrycomics.dangerouscentaur.com" https://E2Q4UU71SRNTMB.cloudfront.net/
dig adamcherrycomics.dangerouscentaur.com @pdns1.namecheap.com +short
dig adamcherrycomics.dangerouscentaur.com @pdns2.namecheap.com +short
nslookup adamcherrycomics.dangerouscentaur.com 8.8.8.8

The Fix: Adding an Explicit CNAME Record

The solution was straightforward: add an explicit CNAME record for adamcherrycomics.dangerouscentaur.com that points directly to the CloudFront distribution. This removes the shadowing issue by making the record explicitly present at the authoritative nameserver.

Infrastructure change at Namecheap:

  • Domain: dangerouscentaur.com
  • Record type: CNAME
  • Hostname: adamcherrycomics
  • Value: CloudFront distribution endpoint
  • HostId assigned: 511341200

The update was performed by merging the new record with the existing DNS record set via Namecheap's API, ensuring all other records (including the wildcard and www subdomain) remained intact.

Verification and Propagation

Once the record was added to Namecheap, propagation was monitored through several steps:

  • Immediate verification: Queried Namecheap's authoritative nameservers directly to confirm the record was present (HostId 511341200 visible in responses)
  • Authoritative polling: Polled pdns2.namecheap.com repeatedly until resolution succeeded, accounting for a ~30-second propagation delay
  • Public resolver validation: Confirmed resolution across multiple public nameservers (8.8.8.8, 1.1.1.1) and Namecheap's servers
  • HTTPS availability: Polled the subdomain over HTTPS until the server responded successfully
  • Content verification: Final curl request confirmed the page content loaded correctly with a 200 response

Why This Approach

While the wildcard CNAME is elegant for catching subdomains automatically, it has a known limitation in DNS: it cannot apply to subdomains of a name that already exists as a DNS node. Rather than restructuring the entire domain's DNS hierarchy, adding the explicit CNAME record was the fastest, lowest-risk solution. This approach:

  • Preserves the wildcard for other subdomains
  • Does not require DNS infrastructure changes
  • Is immediately reversible if needed
  • Follows DNS best practices for subdomain delegation

Key Learnings

This incident highlights several important DNS design patterns:

  • Wildcard limitations: Wildcards do not match names that have explicit DNS nodes. When using wildcard CNAME records, be aware that any child record (like www.adamcherrycomics) will create a node that blocks the wildcard from applying to that level.
  • Authoritative-first debugging: Querying authoritative nameservers directly bypasses caching and propagation delays, providing ground truth faster than querying recursive resolvers.
  • CloudFront as origin: When using CloudFront, always verify the distribution is healthy separately from DNS. The two can fail independently, requiring different diagnostic approaches.

Monitoring and Prevention

To prevent future incidents, monitoring should include:

  • Synthetic checks for critical subdomains (adamcherrycomics, www, root) resolving and returning HTTP 200
  • Alerts on CloudFront distribution status changes
  • Periodic audits of DNS record configuration, particularly around wildcards and explicit records at the same level

The adamcherrycomics subdomain is now fully operational and should remain stable as long as the CloudFront distribution remains active and the explicit CNAME record remains in Namecheap's authoritative nameserver configuration.