Skip to content

[Bug]: java.io.IOException: Cannot run program ... Argument list too long when generating pex #542

Open
@phoenixuprising

Description

@phoenixuprising

What happened?

I had an existing py_binary I was originally building with rules_python and I switched to using aspect_rules_py to generate a pex file. The py_binary depends on just google-api-core, google-auth, google-cloud-secret-manager, bazel-runfiles, and pyyaml packages, and contained a total of 5 source files. When compiling the pex target, it fails with:

ERROR: /Users/chayes/Development/project_root/foo/bar/BUILD:75:21: Building PEX binary //foo/bar:baz_pex failed: (Exit -1): pex failed: error executing PyPex command (from target //foo/bar:baz_pex) bazel-out/darwin_arm64-opt-exec-ST-d57f47055a04/bin/external/aspect_rules_py+/py/tools/pex/pex ... (remaining 19428 arguments skipped)
...
Action failed to execute: java.io.IOException: Cannot run program "/usr/bin/sandbox-exec" (in directory "/private/var/tmp/_bazel_chayes/8fa303648e57c6deaed2bf40b3eacda1/sandbox/darwin-sandbox/3/execroot/_main"): error=7, Argument list too long

When inspecting the logs, the arguments fed into the _pex executable is full of:

'--source=external/+http_archive+gcloud_darwin_arm64/google-cloud-sdk/platform/gsutil/third_party/urllib3/docs/reference/urllib3.fields.rst=+http_archive+gcloud_darwin_arm64/google-cloud-sdk/platform/gsutil/third_party/urllib3/docs/reference/urllib3.fields.rst' '--source=external/+http_archive+gcloud_darwin_arm64/google-cloud-sdk/platform/gsutil/third_party/urllib3/docs/reference/urllib3.poolmanager.rst=+http_archive+gcloud_darwin_arm64/google-cloud-sdk/platform/gsutil/third_party/urllib3/docs/reference/urllib3.poolmanager.rst' '--source=external/+http_archive+gcloud_darwin_arm64/google-cloud-sdk/platform/gsutil/third_party/urllib3/docs/reference/urllib3.request.rst=+http_archive+gcloud_darwin_arm64/google-cloud-sdk/platform/gsutil/third_party/urllib3/docs/reference/urllib3.request.rst'

There were 4.4mb worth of args being passed in total. There is probably another bug in here with regards to how it is passing --source for every file of every package (and transitive package) which is ultimately led to the argument list too long error. I'll see if I can sus out that bug in the next few days and provide a repro case.

Version

Development (host) and target OS/architectures:
MacOS
Darwin chayes.local 24.1.0 Darwin Kernel Version 24.1.0: Thu Oct 10 21:05:23 PDT 2024; root:xnu-11215.41.3~2/RELEASE_ARM64_T6031 arm64

Output of bazel --version:
bazel 9.0.0-pre.20250307.1

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file:
1.3.2
Language(s) and/or frameworks involved:
Python

How to reproduce

Any other information?

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions