Glue: derive a default table location on CREATE TABLE (or add a DEFAULT_TABLE_LOCATION attach option)

[djouallah]

Feature / gap

When attached to AWS Glue's Iceberg REST catalog, every CREATE TABLE must carry an explicit WITH ('location' = 's3://…'), or Glue rejects it:

POST …/iceberg/v1/catalogs/<id>/namespaces/<ns>/tables
400 InvalidInputException: "Location information cannot be null while creating an iceberg table"

This was discussed in #1128 (closed with the WITH ('location'…) workaround), where @Tmonster commented:

Hmm, potentially glue catalogs return a default location that can be used when attaching, I'll look into this.

Requesting exactly that: derive a default table location for Glue when the user doesn't pass one — or, failing that, a DEFAULT_TABLE_LOCATION ATTACH option the client can set once.

Why the workaround isn't always available

The WITH ('location'…) workaround assumes the tool emitting the SQL can add the clause. Higher-level tools often can't: dbt-duckdb generates CREATE TABLE … AS from model definitions and has no mechanism to pass per-table options through to an attached Iceberg catalog. Net effect today: a dbt project that runs fine against OneLake / S3 Tables / Polaris / Snowflake Horizon REST catalogs (where the server assigns locations, via stage-create or otherwise) cannot run against Glue at all — Glue both rejects stage-create (400 "Stage create is not supported", so STAGE_CREATE_TABLES false is mandatory) and refuses to assign a location on direct creates. The two behaviors together mean the client is the only place a location can come from.

Repro

duckdb 1.5.4, iceberg extension 7572645 (also current main by code inspection — IcebergTableSet::CreateTable only reads location from the SQL options):

CREATE SECRET glue_secret (TYPE s3, KEY_ID '…', SECRET '…', REGION 'xx-xxxx-1');
ATTACH '<account_id>' AS glue (TYPE iceberg, ENDPOINT_TYPE 'glue', SECRET glue_secret);
CREATE TABLE glue.demo.t (a INTEGER);
-- 400: Location information cannot be null while creating an iceberg table
CREATE TABLE glue.demo.t (a INTEGER) WITH ('location' = 's3://bucket/demo/t');
-- works

Possible shapes

1. On create without a location, call LoadNamespaceMetadata (Glue returns the database LocationUri in the namespace properties when one is set) and default to <namespace location>/<table_name>. This mirrors what Spark's GlueCatalog does.
2. And/or an ATTACH option, e.g. DEFAULT_TABLE_LOCATION 's3://bucket/warehouse', used as <value>/<namespace>/<table_name> whenever location is absent — works even for namespaces without a LocationUri, and is settable from tools that only expose ATTACH-level configuration (dbt-duckdb profiles, for instance).

Happy to test either against a real Glue catalog — public CI repro setup at https://github.com/djouallah/testing-iceberg-rest-catalog.