Prune and verify architectures for rebuilt macOS wheels (#21692)

This is to prune unused architectures from the wheels we rebuild and
"repair" on macOS.

The primary driver is to enable `delocate_wheel`'s architecture
verification (`require_archs`) to ensure all bundled binaries and their
dependencies have matching architectures.

This brings additional benefits:
- the name of rebuilt/repaired wheels now accurately reflect their
  content, (no foreign architecture in them)
- there's no longer a need for renaming repaired wheels, as their names
  are now correctly set in the first place by the `pip` toolchain when
  rebuilding,
- the size of each rebuilt wheel decreases by 10% to 50% - a modest but
  welcome improvement that helps reduce the Agent's software
  distribution costs.

Single-architecture builds are obtained by setting environment variables
in the `build_macos` function:
- `ARCHFLAGS='-arch <target_arch>'`: set effective build output
  architecture (bundled binaries),
- `_PYTHON_HOST_PLATFORM=macosx-X.Y-<target_arch>`: set wheel platform
  tag (used by the `pip` toolchain to set the wheel's name).

Together, they ensure both binaries and wheel metadata match the single
target architecture, with the former allowing `delocate_wheel` to verify
architecture consistency.

Why environment variables: there is no `--platform` flag for `pip wheel`
to control the build output architecture, because the proposed flag
(pypa/pip#5453) was never merged. Environment variables are the _de
facto_ standard approach used by `cibuildwheel`, `conda-forge` as well
as other wheel building tools.

References:
- pypa/pip#5453: proposal for `--platform` flag (not implemented),
- pypa/cibuildwheel#997: cross-compilation implementation,
- pypa/packaging#882: architecture tag control on macOS,
- https://cibuildwheel.pypa.io/en/stable/platforms/

Author: rdesgroppes

.builders/build.py

@@ -124,6 +124,9 @@ def build_macos():
             'LDFLAGS': f'-L{prefix_path}/lib',
             'CFLAGS': f'-I{prefix_path}/include -O2',
             'CXXFLAGS': f'-I{prefix_path}/include -O2',
+            # Use single-arch builds to avoid bundling extraneous binary payload and enable `require_archs` verification
+            'ARCHFLAGS': f'-arch {os.uname().machine}',
+            '_PYTHON_HOST_PLATFORM': f'macosx-{os.environ["MACOSX_DEPLOYMENT_TARGET"]}-{os.uname().machine}',
             # Build command for extra platform-specific build steps
             'DD_BUILD_COMMAND': f'bash {build_context_dir}/extra_build.sh'
         }

.builders/images/macos/extra_build.sh

@@ -19,7 +19,7 @@ if [[ "${DD_BUILD_PYTHON_VERSION}" == "3" ]]; then
 
     # lmdb doesnt't get the actual full path in its install name which means delocate won't find it
     # Luckily we can patch it here so that it does.
-    install_name_tool -change liblmdb.so "${DD_PREFIX_PATH}/lib/liblmdb.so" "${DD_PREFIX_PATH}/lib//librdkafka.1.dylib"
+    install_name_tool -change liblmdb.so "${DD_PREFIX_PATH}/lib/liblmdb.so" "${DD_PREFIX_PATH}/lib/librdkafka.1.dylib"
     always_build+=("confluent-kafka")
 fi
 

.builders/scripts/repair_wheels.py

@@ -269,17 +269,15 @@ def copy_filt_func(libname):
             shutil.move(wheel, Path(built_dir) / dest)
             continue
 
-        # Platform dependent wheels: rename with single arch and verify target macOS version
-        single_arch = os.uname().machine
-        dest = str(wheel_name._replace(platform_tag=wheel_name.platform_tag.replace('universal2', single_arch)))
+        # Platform dependent wheels: prune excluded files, verify target architecture & macOS version
         copied_libs = delocate_wheel(
             str(wheel),
-            os.path.join(built_dir, dest),
+            os.path.join(built_dir, wheel.name),
             copy_filt_func=copy_filt_func,
-            # require_archs=[single_arch],  TODO(regis): address multi-arch confluent_kafka/cimpl.cpython-312-darwin.so
+            require_archs=[os.uname().machine],
             require_target_macos_version=min_macos_version,
         )
-        print(f'Repaired wheel to {dest}')
+        print('Repaired wheel')
         if copied_libs:
             print('Libraries copied into the wheel:')
             print('\n'.join(copied_libs))