Simplify deployment of ACCP

[cloudshiftchris]

We worked through moving from netty-tcnative to ACCP; it wasn't a smooth path, and still has a few rough edges.

For Netty TcNative we had pulled in the native dependencies via build.gradle.kts as:

    val nettyTcnativeClassifiers = setOf("osx-aarch_64", "osx-x86_64","linux-aarch_64", "linux-x86_64", "windows-x86_64")
    nettyTcnativeClassifiers.forEach { tcNativeClassifier ->
        runtimeOnly("io.netty:netty-tcnative-boringssl-static") { artifact { classifier = tcNativeClassifier } }
    }

Key points:

• the classifiers is the set of all deployment and development classifiers;
• the scope is runtimeOnly as the native libraries only need to be pulled in at runtime;
• the dependency version is resolved via Netty BOM (in this case, via Spring Boot), preventing version mismatch between native dependencies and their Java peers;

Netty has collaborating components - code in Netty itself and these optional dependencies - making it smart enough to attempt to load the (optional) native library for the current platform. For example, libnetty_tcnative_osx_aarch_64.jnilib is included in the osx-aarch_64 classifier. Note that the library name includes the platform information.

We played with a few options for ACCP:

1. Create an Uber JAR (combining the JARs for different classifiers that are identical other than their included native library). This had challenges:

• breaks the JAR signing (only for Oracle JDK, not material for us);
• collisions in native library names as they don't include platform information, e.g. libamazonCorrettoCryptoProvider.dylib is the same name for x86_64 and aarch64 variants;

That was a non-starter.

2. Put logic in build.gradle.kts to account for development and deployment requirements:

    // we currently deploy on AMD64 architecture
    val accpDeploymentClassifier = "linux-x86_64"

    val accpClassifier =
        when {
            System.getenv("CI") == "true" -> accpDeploymentClassifier

            // accp does not have a windows artifact; use the deployment classifier so we don't have
            // resolution / compilation issues, but don't initialize in code
            osdetector.os == "windows" -> accpDeploymentClassifier

            // run locally (likely MacOS)
            else -> osdetector.classifier
        }

    implementation(libs.amazonCorrettoCryptoProvider) { artifact { classifier = accpClassifier } }

This works but isn't equivalent to what we had with Netty:

• its considerably more verbose;
• it hard-codes the deployment target architecture (no where else do we have that), as only a single ACCP library can be included; this precludes using the same build in different deployment targets;
• ACCP needs to be a compile-time dependency (due to the intialization steps), however there is no ACCP library for Windows, so we hack around that;
• it requires the OS Detector plugin (that we otherwise have never needed to use)

Ideally ACCP would support something like:

    // BOM to keep versions aligned
    implementation(platform("software.amazon.cryptools:AmazonCorrettoCryptoProvider-bom:2.4.1"))

    // the Java classes
   implementation("software.amazon.cryptools:AmazonCorrettoCryptoProvider-core")

   // all native dependencies (a JAR that depends on all the individual native jars to pull them all in)
   runtimeOnly("software.amazon.cryptools:AmazonCorrettoCryptoProvider-native-all") 

   // or, for those that wish to selectively pull in platform dependencies
   runtimeOnly("software.amazon.cryptools:AmazonCorrettoCryptoProvider-native")  { artifact { classifier = "linux-x86_64" } }

Aware that ACCP can be deployed out-of-band (into the class path, or installed in the JDK) but that poses additional complexities:

• its uncommon to drop libs in the class path at deployment time (we have no other apps that do this, and frown on the practice) - why would this lib require that?
• the challenges in versioning / resolving / managing deployment outside of the regular build process are problematic;

Perhaps similar to #366, with more color.

[WillChilds-Klein]

Thanks for the report @cloudshiftchris. Just so we understand the ask correctly, are you looking for an "uberjar" style solution that bundles everything together (with distinct names per native lib build), the ability to specify individual components at run/compile time, or both?

[cloudshiftchris]

@WillChilds-Klein the "uberjar" that automagically loads the correct library (or not, with graceful fallback for unsupported platforms) is the primary ask.
The ability to specify select platforms may be of use to some, though that is more specialized and unclear what problem the selectivity actually solves if the "automagic" loading is already there.

The goal being to do reduce barriers to adoption / extensibility - do the minimal work in the consumer's build script (drop in the uberjar), which provides the flexibility to move between platforms w/o rebuilding (or even thinking about all that), no different than any other lib (i.e. hide the "native/platform" details away...).

[matf]

Would appreciate this as well, use case:

• We build libraries that provide encryption/decryption based on ACCP
• These libraries are used by teams on different platforms/architectures (i.e. ECS tasks using x86 or aarch runtimes)
• Because ACCP is platform-dependent, we cannot just include it as a dependency
• Including all available architecture qualifier variants doesn't work either as the ACCP loader simply loads the first librariy found on the classpath, so whether the found library matches the architecture or not is dependent on classpath ordering.

Even solving the last point would already go a long way. Would you be open for a PR to address this?