Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for building iOS wheels. #2286

Merged
merged 40 commits into from
Mar 11, 2025
Merged
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
acf4eb2
Add support for building iOS wheels.
freakboy3742 Feb 17, 2025
e992cef
Replace use of system() in test binary module.
freakboy3742 Feb 25, 2025
a450fba
Restored the 'minimal' approach of the minimal examples.
freakboy3742 Feb 25, 2025
ac8ab09
Split out platform details into standalone pages, and expand iOS plat…
freakboy3742 Feb 25, 2025
254b909
More doc corrections.
freakboy3742 Feb 25, 2025
a7a7b67
Merge branch 'main' into ios-support
freakboy3742 Feb 25, 2025
cead9fb
Bump support package to include fix for python/cpython#130292
freakboy3742 Feb 25, 2025
7b8b31e
Ensure iOS tests are all run on the same xdist worker.
freakboy3742 Feb 26, 2025
9c495a8
More iOS documentation tweaks.
freakboy3742 Feb 26, 2025
b622638
Merge branch 'main' into ios-support
freakboy3742 Mar 2, 2025
457d46a
Factor out common xcode version test utility.
freakboy3742 Mar 2, 2025
05ac4ae
Simplify iOS to a single platform with an expanded interpretation of …
freakboy3742 Mar 2, 2025
3b37676
I guess I should update the iOS tests as well...
freakboy3742 Mar 2, 2025
32b9203
Additional safety for missing iOS test output.
freakboy3742 Mar 2, 2025
c19bd4a
Remove DYLD_LIBRARY_PATH from the iOS environment.
freakboy3742 Mar 3, 2025
cc51dcc
Make test-sources mandatory for iOS builds.
freakboy3742 Mar 5, 2025
be267bd
Updates and clarifications to documentation.
freakboy3742 Mar 5, 2025
5420faf
Clarify what a slice is.
freakboy3742 Mar 6, 2025
fcf6f54
Normalize use of underscores in platform name.
freakboy3742 Mar 6, 2025
c4c88a7
Modify auto target to be matching CPU only.
freakboy3742 Mar 6, 2025
b542fb5
Use consistent ordering of platforms in examples.
freakboy3742 Mar 6, 2025
361b31a
Use consistent naming in iOS archiectures.
freakboy3742 Mar 6, 2025
a995526
Placate the linter.
freakboy3742 Mar 6, 2025
dca2338
Miscellaneous cleanups picked up by @joerick's review.
freakboy3742 Mar 6, 2025
8f2cb91
Correct the list of expected wheels.
freakboy3742 Mar 6, 2025
26f217c
Correct which 'native' we're actually checking.
freakboy3742 Mar 6, 2025
039c300
Correct the docs links so they're all relative.
freakboy3742 Mar 7, 2025
5208fc4
Correct the identification of free threaded builds.
freakboy3742 Mar 7, 2025
eedac55
Use target instead of host to describe the platform we're building for.
freakboy3742 Mar 8, 2025
b4e4a28
Rework iOS test to remove issue with log completeness.
freakboy3742 Mar 8, 2025
e477026
Convert errors to FatalError
freakboy3742 Mar 8, 2025
2ebefab
Removed a repeated check for a valid python.
freakboy3742 Mar 8, 2025
2b08934
Update bin/update_pythons.py to update iOS support packages.
freakboy3742 Mar 9, 2025
3580d6e
Document that iOS CI is available on other platforms.
freakboy3742 Mar 10, 2025
b39c6eb
Restore a comment needed for some platforms.
freakboy3742 Mar 10, 2025
6b9b033
Small cleanups identified in code review
freakboy3742 Mar 10, 2025
f29439b
Simplify logic to appease linter.
freakboy3742 Mar 11, 2025
1ea0fab
Merge branch 'main' into ios-support
freakboy3742 Mar 11, 2025
4a5fef5
Modify dependency constraint handling to use new API.
freakboy3742 Mar 11, 2025
6a8f410
Cosmetic change to trigger a CI rebuild.
freakboy3742 Mar 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use target instead of host to describe the platform we're building for.
freakboy3742 committed Mar 8, 2025
commit eedac55c536a1f5a41099061753b1907e58c560a
58 changes: 30 additions & 28 deletions cibuildwheel/ios.py
Original file line number Diff line number Diff line change
@@ -118,7 +118,7 @@ def build_url(config_dict: dict[str, str]) -> str:
return python_configurations


def install_host_cpython(tmp: Path, config: PythonConfiguration, free_threading: bool) -> Path:
def install_target_cpython(tmp: Path, config: PythonConfiguration, free_threading: bool) -> Path:
if free_threading:
msg = "Free threading builds aren't available for iOS (yet)"
raise ValueError(msg)
@@ -142,30 +142,32 @@ def install_host_cpython(tmp: Path, config: PythonConfiguration, free_threading:
def cross_virtualenv(
*,
py_version: str,
host_python: Path,
target_python: Path,
multiarch: str,
build_python: Path,
venv_path: Path,
dependency_constraint_flags: Sequence[PathOrStr],
) -> dict[str, str]:
"""Create a cross-compilation virtual environment.

In a cross-compilation environment, the *host* is the platform you're
targeting, and the *build* is the platform where you're running the
compilation. When building iOS wheels, iOS is the host machine and macOS is
the build machine.
In a cross-compilation environment, the *target* is the platform where the
code will ultimately run, and the *build* is the platform where you're
running the compilation. When building iOS wheels, iOS is the target machine
and macOS is the build machine. The terminology around these machines varies
between build tools (configure uses "host" and "build"; cmake uses "target" and
"build host").

A cross-compilation virtualenv is an environment that is based on the
*build* python (so that binaries can execute); but it modifies the
environment at startup so that any request for platform details (such as
`sys.platform` or `sysconfig.get_platform()`) return details of the host
`sys.platform` or `sysconfig.get_platform()`) return details of the target
platform. It also applies a loader patch so that any virtualenv created by
the cross-compilation environment will also be a cross-compilation
environment.

:param py_version: The Python version (major.minor) in use
:param host_python: The path to the python binary for the host platform
:param multiarch: The multiarch tag for the host platform (i.e., the value
:param target_python: The path to the python binary for the targret platform
:param multiarch: The multiarch tag for the target platform (i.e., the value
of `sys.implementation._multiarch`)
:param build_python: The path to the python binary for the build platform
:param venv_path: The path where the cross virtual environment should be
@@ -185,9 +187,9 @@ def cross_virtualenv(
# Convert the macOS virtual environment into an iOS virtual environment
# using the cross-platform conversion script in the iOS distribution.

# host_python is the path to the Python binary;
# target_python is the path to the Python binary;
# determine the root of the XCframework slice that is being used.
slice_path = host_python.parent.parent
slice_path = target_python.parent.parent
call(
"python",
str(slice_path / f"platform-config/{multiarch}/make_cross_venv.py"),
@@ -197,17 +199,17 @@ def cross_virtualenv(
)

# When running on macOS, it's easy for the build environment to leak into
# the host environment, especially when building for ARM64 (because the
# architecture is the same as the host architecture). The primary culprit
# for this is Homebrew libraries leaking in as dependencies for iOS
# the target environment, especially when building for ARM64 (because the
# build architecture is the same as the target architecture). The primary
# culprit for this is Homebrew libraries leaking in as dependencies for iOS
# libraries.
#
# To prevent problems, set the PATH to isolate the build environment from
# sources that could introduce incompatible binaries.
env["PATH"] = os.pathsep.join(
[
# The host python's binary directory
str(host_python.parent),
# The target python's binary directory
str(target_python.parent),
# The cross-platform environments binary directory
str(venv_path / "bin"),
# Cargo's binary directory (to allow for Rust compilation)
@@ -239,8 +241,8 @@ def setup_python(
raise ValueError(msg)

# An iOS environment requires 2 python installs - one for the build machine
# (macOS), and one for the host (iOS). We'll only ever interact with the
# *host* python, but the build Python needs to exist to act as the base
# (macOS), and one for the target (iOS). We'll only ever interact with the
# *target* python, but the build Python needs to exist to act as the base
# for a cross venv.
tmp.mkdir()
implementation_id = python_configuration.identifier.split("-")[0]
@@ -261,11 +263,11 @@ def setup_python(
f"{build_python.name} not found, has {list(build_python.parent.iterdir())}"
)

log.step(f"Installing Host Python {implementation_id}...")
log.step(f"Installing Target Python {implementation_id}...")
if implementation_id.startswith("cp"):
host_install_path = install_host_cpython(tmp, python_configuration, free_threading)
host_python = (
host_install_path
target_install_path = install_target_cpython(tmp, python_configuration, free_threading)
target_python = (
target_install_path
/ "Python.xcframework"
/ python_configuration.xcframework_slice
/ "bin"
@@ -275,16 +277,16 @@ def setup_python(
msg = "Unknown Python implementation"
raise ValueError(msg)

assert host_python.exists(), (
f"{host_python.name} not found, has {list(host_install_path.iterdir())}"
assert target_python.exists(), (
f"{target_python.name} not found, has {list(target_install_path.iterdir())}"
)

log.step("Creating cross build environment...")

venv_path = tmp / "venv"
env = cross_virtualenv(
py_version=python_configuration.version,
host_python=host_python,
target_python=target_python,
multiarch=python_configuration.multiarch,
build_python=build_python,
venv_path=venv_path,
@@ -356,7 +358,7 @@ def setup_python(
else:
assert_never(build_frontend)

return host_install_path, env
return target_install_path, env


def build(options: Options, tmp_path: Path) -> None:
@@ -410,7 +412,7 @@ def build(options: Options, tmp_path: Path) -> None:
build_options.dependency_constraints.get_for_python_version(config.version),
]

host_install_path, env = setup_python(
target_install_path, env = setup_python(
identifier_tmp_dir / "build",
config,
dependency_constraint_flags,
@@ -508,7 +510,7 @@ def build(options: Options, tmp_path: Path) -> None:
testbed_path = identifier_tmp_dir / "testbed"
call(
"python",
host_install_path / "testbed",
target_install_path / "testbed",
"clone",
testbed_path,
env=build_env,