Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Windows #357

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .bazelignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
e2e/
examples/uv_pip_compile
bazel-rules_py
8 changes: 8 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,16 @@
# Take care to document any settings that you expect users to apply.
# Settings that apply only to CI are in .github/workflows/ci.bazelrc

startup --windows_enable_symlinks

# dev_dependency for rules_rust
build --enable_runfiles

test --test_output=errors

# test //:requirements.update requires access to USERPROFILE envvar on windows
test --test_env=USERPROFILE

# TODO(alex): enable
common --noenable_bzlmod

Expand Down
1 change: 1 addition & 0 deletions BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ compile_pip_requirements(
extra_args = ["--allow-unsafe"],
requirements_in = "requirements.in",
requirements_txt = "requirements.txt",
requirements_windows = "requirements_windows.txt",
)

modules_mapping(
Expand Down
36 changes: 36 additions & 0 deletions WINDOWS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Using rules_py on windows

## Dev dependencies / issues

rattler_install_packages includes some very long filenames, and this package is configured in cargo.toml to fetch via git. If there are errors on retrieval, try running:
```
git config --system core.longpaths true
```

Can we download rattler_install_packages http (archive.zip) instead?

### Output base

rules_rust generates very long paths. Use --user_output_root=d:\b to partially mitigate.

### zstd crate

`Running cargo build script zstd-sys` fails to build on msvc. I'm not sure why. Perhaps though if we can generate this in some other way it can be a non-issue.

```
error occurred: Command "c:/apps/MVS174/VC/Tools/MSVC/14.40.33521/bin/HostX64/x64/cl.exe" "-nologo" "-MD" "-Z7" "-Brepro" "/nologo" "/DCOMPILER_MSVC" "/DNOMINMAX" "/D_WIN32_WINNT=0x0601" "/D_CRT_SECURE_NO_DEPRECATE" "/D_CRT_SECURE_NO_WARNINGS" "/bigobj" "/Zm500" "/EHsc" "/wd4351" "/wd4291" "/wd4250" "/wd4996" "/showIncludes" "/MD" "/Od" "/Z7" "/wd4117" "-D__DATE__=\"redacted\"" "-D__TIMESTAMP__=\"redacted\"" "-D__TIME__=\"redacted\"" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_DISABLE_ASM=" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-FoD:\\b\\jkggyhq2\\execroot\\aspect_rules_py\\bazel-out/x64_windows-fastbuild/bin/external/crate_index__zstd-sys-2.0.9-zstd.1.5.5/zstd-sys_build_script.out_dir\\zstd/lib/dictBuilder\\zdict.o" "-c" "zstd/lib/dictBuilder\\zdict.c" with args "cl.exe" did not execute successfully (status code exit code: 2).
```

## Test cases

```
bazel --output_user_root=d:\b test //...
```

Three test cases are passing:

```
//py/tests/import-pathing:imp_path_can_not_be_absolute PASSED in 0.4s
//py/tests/import-pathing:imp_path_that_breaks_workspace_root PASSED in 0.4s
//py/tests/import-pathing:py_library_import_pathing_test_suite_test_0 PASSED in 0.4s
```
1 change: 1 addition & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ rust_register_toolchains(
edition = RUST_EDITION,
extra_target_triples = [
"x86_64-apple-darwin",
"x86_64-pc-windows-gnu",
],
versions = [RUST_VERSION],
)
Expand Down
2 changes: 1 addition & 1 deletion e2e/smoke/.bazelversion
8 changes: 8 additions & 0 deletions internal_deps.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,11 @@ def rules_py_internal_deps():
sha256 = "0523026398aea9c8b5f7a4a6d5c0829c285b4fbd960c17b5967a369342e21e01",
downloaded_file_path = "sqlparse-0.4.0-py3-none-any.whl",
)

# required on windows
http_file(
name = "tzdata_2024_1",
urls = ["https://files.pythonhosted.org/packages/65/58/f9c9e6be752e9fcb8b6a0ee9fb87e6e7a1f6bcab2cdc73f02bb7ba91ada0/tzdata-2024.1-py2.py3-none-any.whl"],
sha256 = "9068bc196136463f5245e51efda838afa15aaeca9903f49050dfa2679db4d252",
downloaded_file_path = "tzdata-2024.1-py2.py3-none-any.whl",
)
1 change: 1 addition & 0 deletions internal_python_deps.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ def rules_py_internal_pypi_deps(interpreter):
name = "django",
python_interpreter_target = interpreter,
requirements_lock = "//py/tests/virtual/django:requirements.txt",
requirements_windows = "//py/tests/virtual/django:requirements_windows.txt",
)
8 changes: 8 additions & 0 deletions py/private/toolchain/tools.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,14 @@ TOOLCHAIN_PLATFORMS = {
"@platforms//cpu:aarch64",
],
),
"windows_x86_64": struct(
arch = "x86_64",
vendor_os_abi = "pc-windows-msvc",
compatible_with = [
"@platforms//os:windows",
"@platforms//cpu:x86_64",
],
),
}

def _toolchain_impl(ctx):
Expand Down
10 changes: 9 additions & 1 deletion py/tests/virtual/django/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,17 @@ py_unpacked_wheel(
py_package_name = "Django",
)

py_unpacked_wheel(
name = "tzdata_wheel",
src = "@tzdata_2024_1//file",
py_package_name = "tzdata",
)

compile_pip_requirements(
name = "requirements",
requirements_in = "requirements.in",
requirements_txt = "requirements.txt",
requirements_windows = "requirements_windows.txt",
)

py_library(
Expand All @@ -23,7 +30,7 @@ py_library(
# Depend on django, but not at a particular version, any binary/test rules that
# depend on this (directly or transitively) will need to resolve it to a version
# of their choosing.
virtual_deps = ["django"],
virtual_deps = ["django", "tzdata"],
)

py_binary(
Expand All @@ -43,6 +50,7 @@ py_binary(
resolutions = django_resolutions.override({
# ...but replace the resolution of django with a specific wheel fetched by http_file.
"django": "//py/tests/virtual/django:django_wheel",
"tzdata": "//py/tests/virtual/django:tzdata_wheel",
}),
deps = [":proj"],
)
26 changes: 26 additions & 0 deletions py/tests/virtual/django/requirements_windows.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#
# This file is autogenerated by pip-compile with Python 3.9
# by the following command:
#
# bazel run //py/tests/virtual/django:requirements.update
#
asgiref==3.7.2 \
--hash=sha256:89b2ef2247e3b562a16eef663bc0e2e703ec6468e2fa8a5cd61cd449786d4f6e \
--hash=sha256:9e0ce3aa93a819ba5b45120216b23878cf6e8525eb3848653452b4192b92afed
# via django
django==4.2.7 \
--hash=sha256:8e0f1c2c2786b5c0e39fe1afce24c926040fad47c8ea8ad30aaf1188df29fc41 \
--hash=sha256:e1d37c51ad26186de355cbcec16613ebdabfa9689bbade9c538835205a8abbe9
# via -r py/tests/virtual/django/requirements.in
sqlparse==0.4.4 \
--hash=sha256:5430a4fe2ac7d0f93e66f1efc6e1338a41884b7ddf2a350cedd20ccc4d9d28f3 \
--hash=sha256:d446183e84b8349fa3061f0fe7f06ca94ba65b426946ffebe6e3e8295332420c
# via django
typing-extensions==4.8.0 \
--hash=sha256:8f92fc8806f9a6b641eaa5318da32b44d401efaac0f6678c9bc448ba3605faa0 \
--hash=sha256:df8e4339e9cb77357558cbdbceca33c303714cf861d1eef15e1070055ae8b7ef
# via asgiref
tzdata==2024.1 \
--hash=sha256:2674120f8d891909751c38abcdfd386ac0a5a1127954fbc332af6b5ceae07efd \
--hash=sha256:9068bc196136463f5245e51efda838afa15aaeca9903f49050dfa2679db4d252
# via django
8 changes: 8 additions & 0 deletions py/tools/unpack_bin/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,11 @@ multi_arch_rust_binary_release(
os = "linux",
visibility = ["//tools/release:__pkg__"],
)

multi_arch_rust_binary_release(
name = "windows",
src = ":unpack",
os = "windows",
archs = ["x86_64"],
visibility = ["//tools/release:__pkg__"],
)
8 changes: 8 additions & 0 deletions py/tools/venv_bin/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,11 @@ multi_arch_rust_binary_release(
os = "linux",
visibility = ["//tools/release:__pkg__"],
)

multi_arch_rust_binary_release(
name = "windows",
src = ":venv",
os = "windows",
archs = ["x86_64"],
visibility = ["//tools/release:__pkg__"],
)
Loading