Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make tests work with Java 23 #723

Merged
merged 1 commit into from
Jan 14, 2025
Merged

Conversation

snazy
Copy link
Member

@snazy snazy commented Jan 13, 2025

Example stack trace from Java 23:

getSubject is supported only if a security manager is allowed
java.lang.UnsupportedOperationException: getSubject is supported only if a security manager is allowed
	at java.base/javax.security.auth.Subject.getSubject(Subject.java:347)
	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)
	at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3888)
	at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3878)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3666)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:366)
	at org.apache.iceberg.hadoop.Util.getFs(Util.java:56)
	at org.apache.iceberg.hadoop.HadoopOutputFile.fromPath(HadoopOutputFile.java:53)
	at org.apache.iceberg.hadoop.HadoopFileIO.newOutputFile(HadoopFileIO.java:97)
	at org.apache.polaris.service.dropwizard.catalog.io.TestFileIO.newOutputFile(TestFileIO.java:114)
	at org.apache.iceberg.BaseMetastoreTableOperations.writeNewMetadata(BaseMetastoreTableOperations.java:155)
	at org.apache.iceberg.BaseMetastoreTableOperations.writeNewMetadataIfRequired(BaseMetastoreTableOperations.java:150)
	at org.apache.polaris.service.catalog.BasePolarisCatalog$BasePolarisTableOperations.doCommit(BasePolarisCatalog.java:1343)
	at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:125)
	at org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:201)
	at org.apache.iceberg.rest.CatalogHandlers.createTable(CatalogHandlers.java:274)
	at org.apache.polaris.service.catalog.PolarisCatalogHandlerWrapper.lambda$createTableDirect$13(PolarisCatalogHandlerWrapper.java:588)
	at org.apache.polaris.service.catalog.PolarisCatalogHandlerWrapper.doCatalogOperation(PolarisCatalogHandlerWrapper.java:517)
	at org.apache.polaris.service.catalog.PolarisCatalogHandlerWrapper.createTableDirect(PolarisCatalogHandlerWrapper.java:588)
	at org.apache.polaris.service.catalog.IcebergCatalogAdapter.createTable(IcebergCatalogAdapter.java:258)
	at org.apache.polaris.service.catalog.api.IcebergRestCatalogApi.createTable(IcebergRestCatalogApi.java:205)
	at org.apache.polaris.service.dropwizard.admin.PolarisOverlappingTableTest.createTable(PolarisOverlappingTableTest.java:61)
	at org.apache.polaris.service.dropwizard.admin.PolarisOverlappingTableTest.testTableLocationRestrictions(PolarisOverlappingTableTest.java:153)

Note that javax.security.auth.Subject#getSubject is deprecated for removal since Java 17.

java.lang.System#initPhase3 shows that setting java.security.manager to allow works around the UOE.

@snazy
Copy link
Member Author

snazy commented Jan 13, 2025

To be done after #469

@snazy snazy force-pushed the make-tests-work-with-java-23 branch from 8ed893a to f2e0529 Compare January 14, 2025 08:48
@snazy
Copy link
Member Author

snazy commented Jan 14, 2025

Background: The whole Java SecurityManager stuff was deprecated in Java 17 via JEP 411 (April 2021). Alternatives (Subject.getSubject() --> Subject current()) have been introduced with Java 18.

JEP 486 is about to permanently disable the security manager, even the workaround to set java.security.manager=allow will not work beginning with Java 24 (the JVM won't start).

There's HADOOP-19212 (opened June 2024) with sadly rather stale PR.

Example stack trace from Java 23:

```
getSubject is supported only if a security manager is allowed
java.lang.UnsupportedOperationException: getSubject is supported only if a security manager is allowed
	at java.base/javax.security.auth.Subject.getSubject(Subject.java:347)
	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)
	at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3888)
	at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3878)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3666)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:366)
	at org.apache.iceberg.hadoop.Util.getFs(Util.java:56)
	at org.apache.iceberg.hadoop.HadoopOutputFile.fromPath(HadoopOutputFile.java:53)
	at org.apache.iceberg.hadoop.HadoopFileIO.newOutputFile(HadoopFileIO.java:97)
	at org.apache.polaris.service.dropwizard.catalog.io.TestFileIO.newOutputFile(TestFileIO.java:114)
	at org.apache.iceberg.BaseMetastoreTableOperations.writeNewMetadata(BaseMetastoreTableOperations.java:155)
	at org.apache.iceberg.BaseMetastoreTableOperations.writeNewMetadataIfRequired(BaseMetastoreTableOperations.java:150)
	at org.apache.polaris.service.catalog.BasePolarisCatalog$BasePolarisTableOperations.doCommit(BasePolarisCatalog.java:1343)
	at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:125)
	at org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:201)
	at org.apache.iceberg.rest.CatalogHandlers.createTable(CatalogHandlers.java:274)
	at org.apache.polaris.service.catalog.PolarisCatalogHandlerWrapper.lambda$createTableDirect$13(PolarisCatalogHandlerWrapper.java:588)
	at org.apache.polaris.service.catalog.PolarisCatalogHandlerWrapper.doCatalogOperation(PolarisCatalogHandlerWrapper.java:517)
	at org.apache.polaris.service.catalog.PolarisCatalogHandlerWrapper.createTableDirect(PolarisCatalogHandlerWrapper.java:588)
	at org.apache.polaris.service.catalog.IcebergCatalogAdapter.createTable(IcebergCatalogAdapter.java:258)
	at org.apache.polaris.service.catalog.api.IcebergRestCatalogApi.createTable(IcebergRestCatalogApi.java:205)
	at org.apache.polaris.service.dropwizard.admin.PolarisOverlappingTableTest.createTable(PolarisOverlappingTableTest.java:61)
	at org.apache.polaris.service.dropwizard.admin.PolarisOverlappingTableTest.testTableLocationRestrictions(PolarisOverlappingTableTest.java:153)
```

Note that `javax.security.auth.Subject#getSubject` is deprecated for removal since Java 17.

`java.lang.System#initPhase3` shows that setting `java.security.manager` to `allow` works around the UOE.
@snazy snazy force-pushed the make-tests-work-with-java-23 branch from d1c93d9 to 04d4875 Compare January 14, 2025 10:50
@snazy snazy merged commit 0a2f9df into apache:main Jan 14, 2025
5 checks passed
@snazy snazy deleted the make-tests-work-with-java-23 branch January 14, 2025 11:10
snazy added a commit to snazy/polaris that referenced this pull request Jan 16, 2025
This is basically the same issue as apache#723, just via `@QuarkusIntegrationTest`s, which spawn a separate process. The (JVM) arguments for this need to be specified in a different way.

This also simplifies the Quarksu service build file a little.
snazy added a commit that referenced this pull request Jan 16, 2025
This is basically the same issue as #723, just via `@QuarkusIntegrationTest`s, which spawn a separate process. The (JVM) arguments for this need to be specified in a different way.

This also simplifies the Quarksu service build file a little.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants