Queen of San Diego — Tech Blog

Under the hood at queenofsandiego.com

```html

Diagnosing Dual-Layer Email Delivery Failures: Gmail Relay Credentials vs. Exchange Online Rejection

A recent ticket revealed two distinct email delivery failures occurring in the same message thread—one at the Gmail relay layer, and another at the recipient's Exchange Online server. This post walks through the systematic diagnosis process and the infrastructure decisions that led to the root causes.

The Problem Statement

Three emails sent through the admin@queenofsandiego.com alias were not delivering. The error messages indicated SMTP authentication failures, but the actual failures occurred at different points in the delivery chain. Debugging required tracing through:

  • Gmail's "Send mail as" alias configuration
  • Amazon SES SMTP credential generation and lifecycle
  • DNS infrastructure (SPF, DKIM, DMARC, custom MAIL FROM)
  • Recipient mail server verification and suppression lists

Technical Diagnosis Process

Layer 1: Gmail Relay Credentials

The investigation began by understanding how admin@queenofsandiego.com was configured as a "Send mail as" alias within the jadasailing@gmail.com account. Gmail's alias feature allows sending from alternate addresses by relaying through an external SMTP server—in this case, Amazon SES.

Commands executed to verify SES infrastructure:

# List all SES verified identities and their verification status
aws ses list-identities --region us-east-1

# Get detailed identity configuration
aws ses get-identity-dkim-attributes --identities admin@queenofsandiego.com --region us-east-1

# Verify DKIM CNAME records were properly published
dig _dkim.queenofsandiego.com CNAME
dig token1._domainkey.queenofsandiego.com CNAME
dig token2._domainkey.queenofsandiego.com CNAME
dig token3._domainkey.queenofsandiego.com CNAME

The root cause of Failure 1 was identified: the SMTP credentials stored in Gmail's "Send mail as" configuration for the SES relay had expired or been rotated. Gmail was unable to authenticate to email-smtp.us-east-1.amazonaws.com:587 using the stored username and password.

Why SES SMTP Credentials Expire

AWS SES SMTP credentials are derived from IAM access keys using the AWS Signature Version 4 algorithm. Unlike static passwords, these credentials are computed deterministically from the IAM key material. If an IAM user's access key is rotated (a security best practice), the SMTP credentials automatically become invalid. The derivation process:

  • Converts the IAM secret access key into a message authentication code (HMAC-SHA256)
  • Creates a fixed SMTP password that is valid only for the specific AWS region and IAM identity
  • If the underlying IAM key rotates, the computation produces a different result

This is intentional—it ties SMTP access to the lifetime of the IAM credential.

Layer 2: Exchange Online Rejection

For emails that did successfully send via SES (using the API, not Gmail relay), the next failure occurred at the recipient's mail server. Commands to investigate:

# Check SES suppression list for bounces and complaints
aws ses list-suppressed-destinations --region us-east-1 \
  --reason-to-filter Bounce

# Check specific recipient in suppression list
aws ses list-suppressed-destinations --region us-east-1 \
  --email-address recipient@steigerwald-dougherty.com

# Verify SPF, DKIM, and DMARC DNS records
dig queenofsandiego.com TXT | grep -i spf
dig queenofsandiego.com TXT | grep -i dmarc
dig mail._domainkey.queenofsandiego.com CNAME

The SES suppression list revealed that certain recipients had previously bounced or generated complaints. However, the DNS infrastructure for queenofsandiego.com was correctly configured:

  • SPF record: Included include:amazonses.com to authorize SES as a sending provider
  • DKIM: All three CNAME tokens properly published in DNS
  • DMARC: Policy configured to quarantine messages failing DMARC alignment
  • Custom MAIL FROM: Set to mail.queenofsandiego.com with a custom CNAME

The Exchange Online rejection was not due to authentication failures—the DKIM signatures validated correctly. Rather, it appeared to be a policy-level rejection at the recipient's mail server, potentially triggered by suppression list entries or recipient-side filtering rules.

Infrastructure Review

SES Configuration

The SES setup uses two regions:

  • us-east-1 (primary): Contains the verified identity admin@queenofsandiego.com and hosts the SMTP endpoint
  • us-west-2 (secondary): Hosts Lambda functions for blast campaign delivery (jada-blast-scheduler and jada-blast-sender)

The Lambda functions in us-west-2 communicate with the us-east-1 SES endpoint via API calls, not SMTP. Gmail's relay, however, must connect to an SMTP endpoint in a single region. This architectural choice meant Gmail was hardcoded to use us-east-1.

DNS and Email Authentication

All DNS records were verified to be correctly published:

# Comprehensive DNS check for email authentication
nslookup -type=TXT queenofsandiego.com
dig queenofsandiego.com MX
dig steigerwald-dougherty.com MX  # Check recipient's MX records

No issues were found at the DNS layer. SPF, DKIM, and DMARC were all correctly configured and validated by external tools.

Key Decisions and Trade-offs

Decision 1: Why Use SES SMTP for Gmail Relay?

Using Amazon SES as the SMTP backend for Gmail's "Send mail as" feature provides several advantages:

  • Centralized logging: All outbound email passes through AWS infrastructure, providing unified CloudTrail and SES event publishing
  • Suppression list management: Bounce and complaint handling is automatic via SES, preventing future sends to invalid addresses
  • Domain reputation: Leverage AWS's sending reputation rather than relying solely on Gmail's infrastructure

However, this introduces a credential management burden: the SMTP credentials must be kept in sync with IAM key rotations.

Decision 2: Multi-Region Lambda for Blast Campaigns

Blast campaigns are scheduled in us-west-2 for geographic proximity and isolation from the primary email identity in us-east-1. This allows parallel processing without competing for SES sending limits in a single region.

However, it means that Lambda functions must cross-region call the SES API in us-east-1 to use the verified identity. This adds latency but provides better blast throughput.

Remediation Steps

To resolve Failure 1 (