-
Notifications
You must be signed in to change notification settings - Fork 31
feat(server): next-page prefetch cache for paginated SELECT queries #391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 19 commits
b8194f5
bc3c983
eb9ac07
563e3f8
77195b9
268cdaa
e953c30
73a4d18
a2a9726
1d3d878
59ca642
8c71a39
0293f8f
fab19cd
76f8de7
27a02cd
37a84bf
065b101
d7cf13c
4ba9354
3a93056
c0da110
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -101,6 +101,14 @@ jobs: | |
| -e JAVA_TOOL_OPTIONS="-Dojp.server.slowQuerySegregation.enabled=true" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| # Pagination-cache integration tests run against this server (port 10594) | ||
| - name: Start OJP Server container (prefetch cache on port 10594) | ||
| run: | | ||
| docker run -d --name ojp-server-prefetch-cache \ | ||
| --network host \ | ||
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10594 -Dojp.prometheus.port=9164 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.server.nextPageCache.enabled=true -Dojp.server.nextPageCache.ttlSeconds=60 -Dojp.server.nextPageCache.prefetchWaitTimeoutMs=5000" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| - name: Wait for ojp-server to start | ||
| run: sleep 10 | ||
|
|
||
|
|
@@ -117,11 +125,15 @@ jobs: | |
| # This is the key test step - only H2 tests are enabled | ||
| # All other database tests are disabled by default | ||
| - name: Test (ojp-jdbc-driver) with H2 only | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableH2Tests=true | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableH2Tests=true -DenableH2PrefetchCacheTests=true | ||
|
|
||
| - name: Show ojp-server.log | ||
| if: always() # ensures it runs even if previous steps fail | ||
| run: docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| run: | | ||
| docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| echo "" | ||
| echo "=== OJP Server (with prefetch cache) log ===" | ||
| docker logs ojp-server-prefetch-cache 2>&1 || echo "ojp-server-prefetch-cache container not found" | ||
|
|
||
| # =========================================================================== | ||
| # JOB 2: PostgreSQL Integration Tests | ||
|
|
@@ -220,6 +232,15 @@ jobs: | |
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10593 -Dojp.prometheus.port=9163 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.sql.enhancer.enabled=true -Dojp.sql.enhancer.mode=OPTIMIZE -Dojp.sql.enhancer.dialect=POSTGRESQL" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| # Start third OJP server WITH next-page prefetch cache enabled | ||
| # Pagination-cache integration tests run against this server (port 10594) | ||
| - name: Start OJP Server container (prefetch cache on port 10594) | ||
|
||
| run: | | ||
| docker run -d --name ojp-server-prefetch-cache \ | ||
| --network host \ | ||
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10594 -Dojp.prometheus.port=9164 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.server.nextPageCache.enabled=true -Dojp.server.nextPageCache.ttlSeconds=60 -Dojp.server.nextPageCache.prefetchWaitTimeoutMs=5000" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| - name: Wait for ojp-server to start | ||
| run: sleep 10 | ||
|
|
||
|
|
@@ -235,7 +256,7 @@ jobs: | |
|
|
||
| # Run PostgreSQL-specific tests with -DenablePostgresTests flag | ||
| - name: Test (ojp-jdbc-driver) with PostgreSQL enabled | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenablePostgresTests=true | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenablePostgresTests=true -DenablePostgresPrefetchCacheTests=true | ||
|
|
||
| # =================================================================== | ||
| # SQL Enhancer Integration Test | ||
|
|
@@ -267,6 +288,9 @@ jobs: | |
| echo "" | ||
| echo "=== OJP Server (with SQL enhancer) log ===" | ||
| docker logs ojp-server-enhancer 2>&1 || echo "ojp-server-enhancer container not found" | ||
| echo "" | ||
| echo "=== OJP Server (with prefetch cache) log ===" | ||
| docker logs ojp-server-prefetch-cache 2>&1 || echo "ojp-server-prefetch-cache container not found" | ||
|
|
||
| # =========================================================================== | ||
| # JOB 3: MySQL Integration Tests | ||
|
|
@@ -344,6 +368,14 @@ jobs: | |
| -e JAVA_TOOL_OPTIONS="-Dojp.server.slowQuerySegregation.enabled=true" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| # Pagination-cache integration tests run against this server (port 10594) | ||
| - name: Start OJP Server container (prefetch cache on port 10594) | ||
| run: | | ||
| docker run -d --name ojp-server-prefetch-cache \ | ||
| --network host \ | ||
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10594 -Dojp.prometheus.port=9164 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.server.nextPageCache.enabled=true -Dojp.server.nextPageCache.ttlSeconds=60 -Dojp.server.nextPageCache.prefetchWaitTimeoutMs=5000" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| - name: Wait for ojp-server to start | ||
| run: sleep 10 | ||
|
|
||
|
|
@@ -359,11 +391,15 @@ jobs: | |
|
|
||
| # Run MySQL-specific tests with -DenableMySQLTests flag | ||
| - name: Test (ojp-jdbc-driver) with MySQL enabled | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableMySQLTests=true | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableMySQLTests=true -DenableMySQLPrefetchCacheTests=true | ||
|
|
||
| - name: Show ojp-server.log | ||
| if: always() | ||
| run: docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| run: | | ||
| docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| echo "" | ||
| echo "=== OJP Server (with prefetch cache) log ===" | ||
| docker logs ojp-server-prefetch-cache 2>&1 || echo "ojp-server-prefetch-cache container not found" | ||
|
|
||
| # =========================================================================== | ||
| # =========================================================================== | ||
|
|
@@ -440,6 +476,14 @@ jobs: | |
| -e JAVA_TOOL_OPTIONS="-Dojp.server.slowQuerySegregation.enabled=true" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| # Pagination-cache integration tests run against this server (port 10594) | ||
| - name: Start OJP Server container (prefetch cache on port 10594) | ||
| run: | | ||
| docker run -d --name ojp-server-prefetch-cache \ | ||
| --network host \ | ||
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10594 -Dojp.prometheus.port=9164 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.server.nextPageCache.enabled=true -Dojp.server.nextPageCache.ttlSeconds=60 -Dojp.server.nextPageCache.prefetchWaitTimeoutMs=5000" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| - name: Wait for ojp-server to start | ||
| run: sleep 10 | ||
|
|
||
|
|
@@ -455,11 +499,15 @@ jobs: | |
|
|
||
| # Run MariaDB-specific tests with -DenableMariaDBTests flag | ||
| - name: Test (ojp-jdbc-driver) with MariaDB enabled | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableMariaDBTests=true | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableMariaDBTests=true -DenableMariaDBPrefetchCacheTests=true | ||
|
|
||
| - name: Show ojp-server.log | ||
| if: always() | ||
| run: docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| run: | | ||
| docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| echo "" | ||
| echo "=== OJP Server (with prefetch cache) log ===" | ||
| docker logs ojp-server-prefetch-cache 2>&1 || echo "ojp-server-prefetch-cache container not found" | ||
|
|
||
| # =========================================================================== | ||
| # JOB 5: CockroachDB Integration Tests | ||
|
|
@@ -527,6 +575,14 @@ jobs: | |
| -e JAVA_TOOL_OPTIONS="-Dojp.server.slowQuerySegregation.enabled=true" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| # Pagination-cache integration tests run against this server (port 10594) | ||
| - name: Start OJP Server container (prefetch cache on port 10594) | ||
| run: | | ||
| docker run -d --name ojp-server-prefetch-cache \ | ||
| --network host \ | ||
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10594 -Dojp.prometheus.port=9164 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.server.nextPageCache.enabled=true -Dojp.server.nextPageCache.ttlSeconds=60 -Dojp.server.nextPageCache.prefetchWaitTimeoutMs=5000" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| - name: Wait for ojp-server to start | ||
| run: sleep 10 | ||
|
|
||
|
|
@@ -542,11 +598,15 @@ jobs: | |
|
|
||
| # Run CockroachDB-specific tests with -DenableCockroachDBTests flag | ||
| - name: Test (ojp-jdbc-driver) with CockroachDB enabled | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableCockroachDBTests=true | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableCockroachDBTests=true -DenableCockroachDBPrefetchCacheTests=true | ||
|
|
||
| - name: Show ojp-server.log | ||
| if: always() | ||
| run: docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| run: | | ||
| docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| echo "" | ||
| echo "=== OJP Server (with prefetch cache) log ===" | ||
| docker logs ojp-server-prefetch-cache 2>&1 || echo "ojp-server-prefetch-cache container not found" | ||
|
|
||
| # =========================================================================== | ||
| # JOB 6: DB2 Integration Tests | ||
|
|
@@ -695,6 +755,14 @@ jobs: | |
| -e JAVA_TOOL_OPTIONS="-Dojp.server.slowQuerySegregation.enabled=true" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| # Pagination-cache integration tests run against this server (port 10594) | ||
| - name: Start OJP Server container (prefetch cache on port 10594) | ||
| run: | | ||
| docker run -d --name ojp-server-prefetch-cache \ | ||
| --network host \ | ||
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10594 -Dojp.prometheus.port=9164 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.server.nextPageCache.enabled=true -Dojp.server.nextPageCache.ttlSeconds=60 -Dojp.server.nextPageCache.prefetchWaitTimeoutMs=5000" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| - name: Wait for ojp-server to start | ||
| run: sleep 10 | ||
|
|
||
|
|
@@ -710,11 +778,15 @@ jobs: | |
|
|
||
| # Run DB2-specific tests with -DenableDb2Tests flag | ||
| - name: Test (ojp-jdbc-driver) with DB2 enabled | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableDb2Tests=true | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableDb2Tests=true -DenableDb2PrefetchCacheTests=true | ||
|
|
||
| - name: Show ojp-server.log | ||
| if: always() | ||
| run: docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| run: | | ||
| docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| echo "" | ||
| echo "=== OJP Server (with prefetch cache) log ===" | ||
| docker logs ojp-server-prefetch-cache 2>&1 || echo "ojp-server-prefetch-cache container not found" | ||
|
|
||
| # =========================================================================== | ||
| # JOB 7: Multinode Integration Tests | ||
|
|
@@ -1691,6 +1763,14 @@ jobs: | |
| -e JAVA_TOOL_OPTIONS="-Dojp.server.slowQuerySegregation.enabled=true" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| # Pagination-cache integration tests run against this server (port 10594) | ||
| - name: Start OJP Server container (prefetch cache on port 10594) | ||
| run: | | ||
| docker run -d --name ojp-server-prefetch-cache \ | ||
| --network host \ | ||
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10594 -Dojp.prometheus.port=9164 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.server.nextPageCache.enabled=true -Dojp.server.nextPageCache.ttlSeconds=60 -Dojp.server.nextPageCache.prefetchWaitTimeoutMs=5000" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| - name: Wait for ojp-server to start | ||
| run: sleep 10 | ||
|
|
||
|
|
@@ -1706,11 +1786,15 @@ jobs: | |
|
|
||
| # Run Oracle-specific tests with -DenableOracleTests flag | ||
| - name: Test (ojp-jdbc-driver) with Oracle enabled | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableOracleTests=true | ||
| run: mvn test -pl ojp-jdbc-driver -Dgpg.skip=true -DenableOracleTests=true -DenableOraclePrefetchCacheTests=true | ||
|
|
||
| - name: Show ojp-server.log | ||
| if: always() # ensures it runs even if previous steps fail | ||
| run: docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| run: | | ||
| docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| echo "" | ||
| echo "=== OJP Server (with prefetch cache) log ===" | ||
| docker logs ojp-server-prefetch-cache 2>&1 || echo "ojp-server-prefetch-cache container not found" | ||
|
|
||
| # =========================================================================== | ||
| # JOB 10: SQL Server Integration Tests | ||
|
|
@@ -1794,6 +1878,14 @@ jobs: | |
| -e JAVA_TOOL_OPTIONS="-Dojp.server.slowQuerySegregation.enabled=true" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| # Pagination-cache integration tests run against this server (port 10594) | ||
| - name: Start OJP Server container (prefetch cache on port 10594) | ||
| run: | | ||
| docker run -d --name ojp-server-prefetch-cache \ | ||
| --network host \ | ||
| -e JAVA_TOOL_OPTIONS="-Dojp.server.port=10594 -Dojp.prometheus.port=9164 -Dojp.server.slowQuerySegregation.enabled=true -Dojp.server.nextPageCache.enabled=true -Dojp.server.nextPageCache.ttlSeconds=60 -Dojp.server.nextPageCache.prefetchWaitTimeoutMs=5000" \ | ||
| rrobetti/ojp:0.4.1-SNAPSHOT | ||
|
|
||
| - name: Wait for ojp-server to start | ||
| run: sleep 10 | ||
|
|
||
|
|
@@ -1814,11 +1906,16 @@ jobs: | |
| run: | | ||
| mvn test -pl ojp-jdbc-driver -Dgpg.skip=true \ | ||
| -DenableSqlServerTests=true \ | ||
| -DenableSqlServerPrefetchCacheTests=true \ | ||
| -Dtest="SQLServer*" | ||
|
|
||
| - name: Show ojp-server.log | ||
| if: always() # ensures it runs even if previous steps fail | ||
| run: docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| run: | | ||
| docker logs ojp-server 2>&1 || echo "ojp-server container not found" | ||
| echo "" | ||
| echo "=== OJP Server (with prefetch cache) log ===" | ||
| docker logs ojp-server-prefetch-cache 2>&1 || echo "ojp-server-prefetch-cache container not found" | ||
|
|
||
| # =========================================================================== | ||
| # JOB 11: Notify Integration Repository | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we still need a separated ojp server if now the nextPageCache is enabled in the client side per datasource?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No — the separate server is no longer needed. Since the cache opt-in is now a client-side property (
ojp.nextPageCache.enabled), the main OJP server is all that's needed. Fixed in3a93056: thenextPageCacheflags (enabled,ttlSeconds,prefetchWaitTimeoutMs) are moved to the main server in all 8 CI jobs, theojp-server-prefetch-cachecontainer is removed, and all 7 pagination-cache CSV files are updated from port10594→1059.