S3 Destination Connector

[kmacrow]

✨ Enhancement Request

Summary:
Add an S3 destination connector for structured data.

---

Problem / Use Case:

• I want to send data to an S3 data lake destination

---

Proposed Solution:

• Start with support for traditional Hive partitioning and compressed Parquet files, e.g. s3://my-bucket/<namespace>/<schema>/<table>/date=2025-01-01/
• Add support for additional formats: JSON/NDJSON, ORC, Avro
• Add support for writing transactional table formats: Iceberg, Delta, Hudi, S3 Tables

---

Alternatives Considered:

• Redshift destination connector is capable of writing Parquet to S3 but paths do not follow conventional data lake partitioning etc.

---

Impact / Importance:

• High impact generally, especially for users that need to drop data in secure drop zones on the customer side where data warehouses cannot be exposed and peering or other network federation are not possible

---

Additional Context (optional):

• Part of a series of enhancements on supporting object connectors as sources and destinations