No support for Python packages that rely on manylinux wheels newer than `manylinux2014`

[svet-b]

Description:

Using sam build (without --use-container) and trying to install a recent version of Polars, such as 1.12.0, fails with "Could not find a version that satisfies the requirement ...". Only Polars versions up to 0.19.12 are listed as available.

This is because Polars stopped shipping wheels based on manylinux2014 (link), on the basis that it's outdated (and essentially EOL).

Meanwhile, at present sam build supports only manylinux up to manylinux2014 (aka manylinux_2_17) wheels, and no versions more recent than that (source). I'm experiencing this on ARM64, but it appears to be true for x86_64 also.

I haven't done a thorough search on how widespread of an issue this is, but there's at least one other similar report here: #674

Given that Python package maintainers are moving away from an outdated version of manylinux, it would be great for this toolchain to also support more recent versions.

I should note that using sam build --use-container does build successfully. But I'd like to avoid that since (a) it adds some weight to the build process, and (b) I'm running the build inside CodePipeline, which is not happy with me trying to spin up a container inside it.

Hopefully it's ok to not add further detail for now as I think the issue I'm describing is self-evident - but happy to attach some detailed logs/reproducers if actually helpful.

[mndeveci]

Thanks for reporting this issue.

If you build it with sam build --use-container, does running sam local invoke and deploying the function works as expected? If so, we need to attach this issue to the one above to make sure AWS Lambda Builders can support the newer manylinux versions that is supported by Lambda.

[svet-b]

Thank you for looking into this. Indeed, after building with --use-container, running sam local invoke works as expected.

[vicheey]

Thank you for confirming. We are keeping this issue open for now to see if we have any reaction from community who face similar issue and will prioritize accordingly.

[stschulte]

I saw similar issues trying to use rdkit which prevents using any newer versions. Since I am not using SAM, but use terraform for means of deployment, we use aws-lambda-builder directly to turn a python project to a ZIP file we can deploy to S3. This currently limits us in what depdendencies we can include in our projects.

Is the platform a restriction of the lambda builder or is this a restriction of the runtime? I assumed that latest runtimes like python3.13 would also shop with a recent glibc?

[alesanfra]

Hi there,

I'm Alessio from Verizon, and I'm experiencing the same issue as of today when trying to install numpy==2.3.0, which apparently requires manylinux_2_28.

It would be great if there was an option to override the manylinux version through an environment variable or CLI option. Since the Python 3.12+ Lambda environment runs on AL2023, there shouldn't be any compatibility issues with using manylinux_2_28 with python3.12 and python3.13 runtimes.

Thanks for considering this enhancement!

Alessio

[valerena]

I'll add an update here, because there has been more info in the aws-sam-cli version of this GitHub issue. The last released version (1.55.0, included in SAM CLI 1.142.1) allowed aws-lambda-builders to consider manylinux_2_28 (and other versions up to 2_34) as compatible for purposes of building instead of just checking for a really old version like it was doing before. However, this mostly works when the environment where you're building is similar to the one you're building for.

That means that when trying to build cross architecture, aws-lambda-builders still doesn't manage the new platform values correctly. More details about why this happens are described here: aws/aws-sam-cli#8101 (comment)

We're currently in discussions about how and when we'll get this solved, and we'll post updates when we have more info.