Automating Charter Operations Data Pipeline: From Calendar Events to S3 Publishing

```html

This session focused on building an automated data pipeline to extract charter operations information from JADA's internal calendar system, generate trip documentation, and publish artifacts to S3 for crew access. The goal was to reduce manual data entry and ensure consistent, timely distribution of charter manifests and trip sheets to the operations team.

What Was Done

I built a multi-stage pipeline that:

Extracted weekend charter events from JADA's internal calendar using OAuth-authenticated API calls
Generated HTML manifests and trip sheets by templating calendar data
Published generated documents to S3 for access by crew members and operations staff
Created a comprehensive readiness report documenting all weekend operations
Established a project structure for shipcaptaincrew to manage recurring charter workflows

Technical Details: Calendar Integration

The first challenge was accessing JADA's calendar without hardcoding credentials. The solution leveraged OAuth token refresh patterns already established in the codebase:


# Calendar fetch with OAuth token management
GET /calendar/events?dateRange=2026-05-29..2026-05-31
Authorization: Bearer {refreshed_oauth_token}

Rather than making single API calls, I implemented extended date range queries to capture all charters across a wider window. This proved essential when the initial 3-day window missed charters due to timezone handling or scheduling quirks. The solution expanded queries to 2-week ranges, allowing filtering logic downstream to identify weekend-specific operations.

Key files created during this phase:

/Users/cb/Documents/repos/jada-ops/weekend-charters-readiness-2026-05-29.md — Comprehensive readiness report with all charters, crew assignments, and payment status
/Users/cb/Documents/repos/agent_handoffs/projects/quinn-male-charter.md — Specific charter project file with Quinn Male booking details

Technical Details: Document Generation

Once calendar data was extracted, the pipeline generated HTML documents using template patterns. Two document types were created:

Trip Sheet Template — Contains operational details for crew:


Document structure:
- Charter ID and booking reference
- Passenger manifest (names, payment status)
- Vessel assignment and captain
- Departure/return times with timezone
- Special requests or dietary restrictions
- Emergency contact information
- Payment verification status

Manifest Template — Formal record for operations:


Document structure:
- Company header and charter date
- Passenger list with ID numbers
- Crew roster with role assignments
- Liability waiver tracking
- Equipment checklist status
- Pre-charter safety briefing sign-offs

These were generated as standalone HTML files in /tmp/ during development:

/tmp/quinn-male-manifest.html
/tmp/quinn-male-trip-sheet.html

The rationale for separate documents: trip sheets are crew-facing operational guides, while manifests are formal compliance records. Keeping them separate allows different access controls and retention policies downstream.

Infrastructure: S3 Publishing Strategy

Documents were published to S3 to make them accessible to distributed crew members without requiring direct file access to development machines. The publishing process involved:

AWS credential refresh through existing session management
Validation that generated documents existed in /tmp/ before upload
S3 object creation with appropriate metadata headers
Verification that objects persisted after upload

The S3 publishing workflow encountered authentication challenges—the initial OAuth session had expired. Rather than embedding credentials in code, the solution used the existing AWS session refresh mechanism:


# Reauthenticate AWS session
# This leverages existing ~/.aws/config profile
# and environment-based credential chain
aws sts get-caller-identity  # Verify session validity
aws s3 cp /tmp/quinn-male-manifest.html s3://shipcaptaincrew-docs/manifests/
aws s3 cp /tmp/quinn-male-trip-sheet.html s3://shipcaptaincrew-docs/trips/

Key infrastructure decisions:

S3 bucket structure: Organized by document type (manifests, trips, proposals) to enable role-based access policies
Temporary storage: Using /tmp/ for generated documents ensures they're cleaned up automatically and don't clutter the repo
No CloudFront/Route53 in this phase: Documents are accessed directly from S3 by crew members with IAM permissions; CDN distribution wasn't necessary for this low-volume, low-latency requirement

Project Structure: shipcaptaincrew Repository

To support recurring charter workflows, I established a project directory structure:


/Users/cb/Documents/repos/shipcaptaincrew/
├── tools/
│   ├── calendar-fetch.py          # OAuth calendar extraction
│   ├── manifest-generator.py      # HTML templating
│   └── s3-publisher.py            # Secure S3 upload wrapper
├── templates/
│   ├── manifest.html.jinja        # Manifest template
│   └── trip-sheet.html.jinja      # Trip sheet template
├── config/
│   └── s3-buckets.yaml            # Bucket names and policies
└── docs/
    └── reference-charters/        # Example completed charters

This structure separates concerns: tools are reusable scripts, templates are portable, and config is externalized for different environments (dev, staging, production).

Key Decisions & Rationale

Why OAuth instead of API keys? OAuth tokens can be automatically refreshed without embedding long-lived secrets in code. They're also traceable to specific user sessions, providing better audit trails for compliance.

Why separate manifest and trip sheet documents? Different stakeholders need different information. Crew wants operational details; compliance needs formal records. Separating them reduces data exposure and allows different distribution channels.

Why validate documents after S3 upload? Network failures or permission issues could cause silent upload failures. Verification ensures the pipeline catches errors rather than leaving crew without documentation.

Why store generated HTML in /tmp instead of the repo? Generated artifacts shouldn't be version-controlled. Using /tmp/ keeps the repository clean and prevents merge conflicts on rebuilt files.

What's Next

Automated scheduling: Wrap this pipeline in a cron job or Lambda trigger to run every Friday and generate weekend charters automatically
Email distribution: Integrate with JADA's email system to notify crew when new trip sheets are published
Payment verification: Add a step to cross-reference calendar payment status with actual invoice records before publishing
Template customization: Implement per-vessel or per-captain customizations to the manifest and trip sheet templates
Access logging: Enable S3 access logs to track which crew members downloaded which charters for audit purposes

The foundation is now in place to expand charter operations automation across the entire JADA fleet.

```