Skip to content

Conversation

sunng87
Copy link
Member

@sunng87 sunng87 commented Sep 5, 2025

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

This patch will pull in datafusion-postgres for an enhanced postgres support on datafusion.

Tasks:

  • Apply SQL rewrite rules
  • Add UDFs
  • Add UDTFs
  • Add built-in tables

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.
  • API changes are backward compatible.
  • Schema or data changes are backward compatible.

@github-actions github-actions bot added size/S docs-not-required This change does not impact docs. labels Sep 5, 2025
@github-actions github-actions bot added size/XL and removed size/M labels Sep 23, 2025
@sunng87 sunng87 marked this pull request as ready for review September 24, 2025 00:29
@sunng87 sunng87 requested a review from a team as a code owner September 24, 2025 00:29
@sunng87
Copy link
Member Author

sunng87 commented Sep 24, 2025

This patch has not included all pg_catalog compatibility fixes from datafusion-postgres. We will update again when we upgraded to datafusion 50.

@sunng87
Copy link
Member Author

sunng87 commented Sep 24, 2025

A summary of this patch:

  • replaced all pg_catalog implementation with datafusion-postgres
  • bridge between our API with datafusion-postgres: mostly for reading recordbatches from datafusion tables
  • register UDFs and UDTFs
  • fix an issue with sqlness runner on postgresql output
  • update sqlness results for these tables
  • use datafusion-postgres sql rewriter to rewrite sql into the form that works with datafusion
  • update parser to support sql rewritten by postgres dialect (FETCH n IN CURSOR, show )
  • use patch.crate-io to include our datafusion fork, so it overrides all datafusion 49 in our dependency tree.

@sunng87
Copy link
Member Author

sunng87 commented Sep 25, 2025

image

@yihong0618
Copy link
Contributor

image

amazing!

@sunng87 sunng87 enabled auto-merge September 25, 2025 03:32
@sunng87 sunng87 added this pull request to the merge queue Sep 25, 2025
auto-merge was automatically disabled September 25, 2025 04:15

Pull Request is not mergeable

Merged via the queue into GreptimeTeam:main with commit 964dc25 Sep 25, 2025
40 of 42 checks passed
@sunng87 sunng87 deleted the feature/pg_catalog branch September 25, 2025 04:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-not-required This change does not impact docs. size/XXL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants