Summary
When querying a Delta table on Alibaba Cloud OSS via DuckDB, delta_scan appears to use path-style addressing through DeltaKernel, even though the DuckDB secret is configured with URL_STYLE 'vhost'. OSS forbids path-style access for security reasons, resulting in a 403 error.
Configuration
I created a secret in DuckDB to access OSS with vhost-style:
CREATE OR REPLACE SECRET alicloud_secret (
TYPE S3,
URL_STYLE 'vhost',
KEY_ID 'xxxx',
SECRET 'xxxx',
REGION 'ap-southeast-1',
ENDPOINT 'oss-ap-southeast-1.aliyuncs.com'
);
Then I ran:
select * from delta_scan('s3://bucketname/a/s');
Observed behavior
delta_scan attempts to access the Delta log using path-style addressing, leading to OSS rejecting the request:
"error":"IO Error: DeltKernel ObjectStoreError (8): Error interacting with object store: The operation lacked the necessary privileges to complete for path a/s/_delta_log/_last_checkpoint: Error performing GET https://oss-ap-southeast-1.aliyuncs.com/bucketname/a/s/_delta_log/_last_checkpoint in 682.801917ms - Server returned non-2xx status code: 403 Forbidden: <?xml version=\"1.0\" encoding=\"UTF-8\"?>
<Error>
<Code>SecondLevelDomainForbidden</Code>
<Message>Please use virtual hosted style to access.</Message>
Expected behavior
DuckDB/DeltaKernel should honor URL_STYLE 'vhost' and use virtual-hosted-style requests for OSS:
Expected format: https://bucketname.oss-ap-southeast-1.aliyuncs.com/a/s/_delta_log/_last_checkpoint
Not: https://oss-ap-southeast-1.aliyuncs.com/bucketname/a/s/_delta_log/_last_checkpoint
Questions
How can I configure DuckDB so that delta_scan (via DeltaKernel) uses vhost-style addressing on OSS?
Summary
When querying a Delta table on Alibaba Cloud OSS via DuckDB, delta_scan appears to use path-style addressing through DeltaKernel, even though the DuckDB secret is configured with URL_STYLE 'vhost'. OSS forbids path-style access for security reasons, resulting in a 403 error.
Configuration
I created a secret in DuckDB to access OSS with vhost-style:
Then I ran:
Observed behavior
delta_scan attempts to access the Delta log using path-style addressing, leading to OSS rejecting the request:
Expected behavior
DuckDB/DeltaKernel should honor URL_STYLE 'vhost' and use virtual-hosted-style requests for OSS:
Expected format: https://bucketname.oss-ap-southeast-1.aliyuncs.com/a/s/_delta_log/_last_checkpoint
Not: https://oss-ap-southeast-1.aliyuncs.com/bucketname/a/s/_delta_log/_last_checkpoint
Questions
How can I configure DuckDB so that delta_scan (via DeltaKernel) uses vhost-style addressing on OSS?