ordeq-iceberg: Add Ordeq Iceberg package#416
Open
Josersanvil wants to merge 38 commits intoing-bank:mainfrom
Open
ordeq-iceberg: Add Ordeq Iceberg package#416Josersanvil wants to merge 38 commits intoing-bank:mainfrom
ordeq-iceberg: Add Ordeq Iceberg package#416Josersanvil wants to merge 38 commits intoing-bank:mainfrom
Conversation
ordeq-iceberg: Add Ordeq Iceberg package
There was a problem hiding this comment.
Pull request overview
This pull request introduces the initial implementation of the ordeq-iceberg package, providing integration between Ordeq and Apache Iceberg. The package enables working with Iceberg catalogs and tables through Ordeq's IO abstractions, with support for loading catalogs, reading tables, and creating tables with configurable behavior.
Key changes:
- Core IO classes (
IcebergCatalog,IcebergTable,IcebergTableCreate) for Iceberg integration - Custom error handling with
IcebergIOErrorandIcebergTableAlreadyExistsError - Comprehensive test suite including unit tests and resource-based integration tests
Reviewed changes
Copilot reviewed 14 out of 16 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
src/ordeq_iceberg/__init__.py |
Exports main IO classes and error types for package API |
src/ordeq_iceberg/catalog.py |
Implements IcebergCatalog for loading Iceberg catalogs with enum/string support |
src/ordeq_iceberg/table.py |
Implements IcebergTable for loading tables from catalogs |
src/ordeq_iceberg/table_create.py |
Implements IcebergTableCreate with configurable if-exists behavior |
src/ordeq_iceberg/errors.py |
Defines custom exceptions for Iceberg operations |
src/ordeq_iceberg/py.typed |
Marks package as type-hinted |
pyproject.toml |
Configures build system, dependencies, and tooling for the package |
tests/conftest.py |
Provides reusable pytest fixtures for catalog and resource directory setup |
tests/test_catalog.py |
Tests catalog loading with different type formats |
tests/test_table.py |
Tests table loading from catalogs |
tests/test_table_create.py |
Tests table creation with various if-exists options |
tests/test_resources.py |
Snapshot testing for resource examples |
tests/resources/iceberg_test/create_and_load.py |
Example demonstrating table creation and loading workflow |
tests/snapshots/iceberg_test/create_and_load.snapshot.md |
Expected output snapshot for the example resource |
packages/ordeq-iceberg/tests/resources/iceberg_test/create_and_load.py
Outdated
Show resolved
Hide resolved
…e in create_save_table function
sbrugman
reviewed
Nov 24, 2025
sbrugman
reviewed
Nov 24, 2025
…atting, and refine test fixtures
… and schema; update tests for schema handling
ff91f01 to
70945a8
Compare
…date related logic
…remove IcebergTableCreate class
…r if_exists option
…stead of separate table_name and namespace attributes
f8f7f31 to
d7a5699
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copilot summary
This pull request introduces a new Ordeq integration package for Apache Iceberg, named
ordeq-iceberg. The package provides IO classes for loading Iceberg catalogs and tables, and includes supporting configuration, type hints, and a comprehensive test suite. The changes also register the new package in the workspace.New Ordeq-Iceberg Package:
ordeq-icebergto the workspace and created itspyproject.tomlwith metadata, dependencies (includingordeq,pyarrow, andpyiceberg), build settings, test dependencies, and code coverage configuration. [1] [2]ordeq_iceberg/__init__.pyto expose the main IO classes:IcebergCatalogandIcebergTable.py.typedfile to indicate type hints are provided.Core Functionality:
IcebergCatalogas a dataclass for loading Iceberg catalogs, supporting both enum and string catalog types.IcebergTableas a dataclass for loading Iceberg tables from a catalog, handling both direct and IO-catalog inputs.Testing and Examples:
test_catalog.py.test_table_load.py.