[SYCL][E2E] Add logic to react to `REQUIRED`/`UNSUPPORTED` in `build-only` #16725

ayylol · 2025-01-21T21:08:38Z

Adds logic for evaluating REQUIRES/UNSUPPORTED statements on build-only mode by "ignoring" features in these statements that do not affect the compilation (Everything other than OS, triple, and SDK/libraries).

More precisely, the ability to ignore features is implemented by extending the boolean expressions to use a third boolean state - ignore. If a particular sub-expression includes an ignore, and if its result could be changed by setting that ignore to either true or false, then the result of that sub-expression is set to ignore. For example ignore || true = true, but ignore || false = ignore, this is because in the first example there would be no way to set the ignore such that the result is anything other than true, while in the second example the result is dependent on the "actual" value of the ignore.

If the resulting value of a REQUIRES predicate is ignore we interpret that as true (The requirements were met), on the other hand for UNSUPPORTED predicates we would interpret ignore as false instead (The unsupported criteria was not met).

The triples that can be used for a given test are then selected by evaluating the REQUIRES/UNSUPPORTED with the set of features + the target feature corresponding to each triple (mirroring the way we select devices), while ignoring all features that do not affect compilation.

The target features available are the following: target-spir, target-amd, target-nvidia, and target-native_cpu. Each of these map to a triple.

Similarly to how XFAIL is handled when using multiple devices if we are compiling for multiple triples, and a single triple is marked as XFAIL, then it is treated as unsupported instead.

The available target triples in build-only mode is determined through the use of the sycl_build_targets lit param (i.e., --param sycl_build_targets=spir;amd). This is currently set to only spir, as the changes in test markup included in this pr do not take into account building for nvptx and amdgcn triples. In run-only and full mode the available triples are determined via the available devices.

ayylol · 2025-01-21T21:12:01Z

This pr is the logic from #16637, but only with support for spir64 triple, that way 205 file changes do not need to reviewed at once. Changes to the test markup to support amd and nvidia will be made in future prs.

sycl/test-e2e/TriStateExpr.py

aelovikov-intel · 2025-01-21T21:21:10Z

sycl/test-e2e/TriStateExpr.py

+        self.assertTrue(
+            E2EExpr.evaluate("!rt_feature", {"rt_feature"}, True, True)
+        )
+        self.assertTrue(
+            E2EExpr.evaluate("!!rt_feature", {"rt_feature"}, True, True)
+        )


IMO, there are far too many tests with {"rt_feature"} when run time features are to be ignored. We don't expect the set of "set" features to contain them, so why are we testing that scenario mostly?

I can understand having a couple of tests for completeness/to document behavior, but having too many is counterproductive and might confuse the reader about the purpose of this class.

Instead of deleting these, I just made the available features list empty. Does that work? otherwise I can delete them.

My intention with these was to show that ignore values propagate up the expression.

sycl/test-e2e/format.py

sarnex

quick initial review, will look more indepth once andrei's feedback is addressed

sycl/test-e2e/TriStateExpr.py

sarnex · 2025-01-21T21:43:54Z

sycl/test-e2e/format.py

+            if self.getMissingRequiredFeaturesFromList(features, test.requires, build_only_mode=True):
+                continue
+            if self.getMatchedFromList(features, test.unsupported, build_only_mode=True):


these functions are called get but are being used as has it seems, should we rename?

hasMissingRequiredFeatures and hasUnsupportedFeatures? or what were you thinking. One thing to note that brought some confusion in the past is the fact that these dont report the specific features that were missing to pass the expression, rather it returns the clauses that did not pass.

Also should note that getMissingRequiredFeaturesFromList is named after a function of the same name in the lit implementation. As for getMatchedFromList that one doesnt have an exact corresponding function, but there is getUnsupportedFeatures which is similar, but does not have an argument to pass in the features available.

do we ever use the return value's contents or is it just empty or not? if its only empty or not i think we should rename to has..., otherwise current is fine

One small caveat, the getMatchedFromList function is also used for determining XFAILs, so im not entirely sure hasUnsupportedFeatures would be entirely accurate.

This False implies that we're processing UNSUPPORTED, we need to modify the name of this function to indicate that now. Does lit.Test have some name that we can copy?

@aelovikov-intel tagging you here, because I think these comments are asking for the same thing, going to mark the other as resolved and we can continue discussion here

IMO, keeping it close to lit.Test or whatever the names came from is more important than renaming just because we don't use the result.

One small caveat, the getMatchedFromList function is also used for determining XFAILs, so im not entirely sure hasUnsupportedFeatures would be entirely accurate.

In this case, I'd avoid hardcoding False in the body and pass it as an additional parameter. Yes, it's kind of useless, but it would make interface more consistent.

yeah im not super familiar with the implementation of lit so if the function names are standard then its fine

sycl/test-e2e/format.py

sycl/test-e2e/lit.cfg.py

sycl/test-e2e/TriStateExpr.py

sycl/test-e2e/format.py

Maetveis · 2025-01-22T08:42:48Z

sycl/test-e2e/lit.cfg.py

+    target = config.backend_to_target[be]
+    features.add(target)
+    config.sycl_triples.add(target)


Backend is the underlying runtime that sycl is implemented on, right? Isn't it possible to have the same backend support multiple targets?. For example using OpenCL on AMD or NVIDIA (theoretically of course since practically they would have to support the extensions sycl requires).

I think if backend_to_target would be backend_default_target then it leaves the design open to add an override via some option like --param backend_targets_<be>=<target1>,<target2>.
Or maybe this is what architectures is for?

I think if backend_to_target would be backend_default_target then it leaves the design open to add an override via some option like --param backend_targets_<be>=<target1>,<target2>.
Or maybe this is what architectures is for?

Yeah, something like that could be possible to do by overriding/changing the dictionary used for the mappings. Although I do still prefer the x_to_y naming scheme for these mapping dictionaries, since it groups them together with consistent naming, and I don't think it necessarily stops us from redefining them in the future through parameters like you suggest.

Maetveis · 2025-01-22T08:47:00Z

sycl/test-e2e/lit.cfg.py

+config.target_to_triple = {
+    "target-spir": "spir64",
+    "target-nvidia":"nvptx64-nvidia-cuda",
+    "target-amd":"amdgcn-amd-amdhsa",
+    "target-native_cpu":"native_cpu",
+}
+config.triple_to_target = {v:k for k,v in config.target_to_triple.items()}


What's the difference between a "target" and the "triple" it corresponds to, given there's a 1:1 mapping between them? Is target just a short name for the triple? Is it an alias to allow changing the target triples without updating all the tests that refer to it?

Short name was part of the motivation (at least for me). Also, Nick mentioned that there is some work to have two different spir triples (spir64 and spirv, I think).

aelovikov-intel

IMO, we it's good to be merged as-is. Any updates fixes can (and probably should) be done in subsequent PRs.

ayylol · 2025-01-24T20:42:19Z

Hi @Maetveis, thank you for your comments and concerns. If you had any more concerns related to this I am going to continue to work on this after this pr, so I will be ready to work with you to address any other concerns in the post-commit.

Also wanted to check in with @uditagarwal97 to ask if you are good to continue with these changes as well.

uditagarwal97

Changes looks reasonable to me.

… tests (#16787) As of #16725 tests that do not build for `spir64` do not need to be marked as exceptions for split build/run with `REQUIRES: build-and-run-mode`, instead we can mark them as `REQUIRES: target-<nvidia/amd>`. This patch replaces the `REQUIRES: build-and-run-mode` directives in Matrix tests with `REQUIRES: target-amd` or `REQUIRES: target-nvidia`.

…nd `Adapters` tests (#16795) As of #16725 tests can properly react to features that affect compilation on build-only mode (i.e., libraries or OS), additionally we can also mark if a test should only be built for a specific triple using the `target-*` features. This pr removes `REQUIRES: build-and-run-mode` from syclcompat and Adapters tests, and either lets the test be marked as unsupported due to requiring a missing build feature (`windows` or `cuda_dev_kit`), or the test is marked as unsupported for `spir` by requiring `target-nvidia`

Add method to set triple + react to REQUIRES in build-only

123b4ac

ayylol requested review from sarnex, Maetveis, uditagarwal97 and aelovikov-intel January 21, 2025 21:08

ayylol requested review from a team as code owners January 21, 2025 21:08

ayylol temporarily deployed to WindowsCILock January 21, 2025 21:09 — with GitHub Actions Inactive

aelovikov-intel reviewed Jan 21, 2025

View reviewed changes

sycl/test-e2e/TriStateExpr.py Outdated Show resolved Hide resolved

aelovikov-intel reviewed Jan 21, 2025

View reviewed changes

sycl/test-e2e/format.py Outdated Show resolved Hide resolved

ayylol marked this pull request as draft January 21, 2025 21:34

aelovikov-intel reviewed Jan 21, 2025

View reviewed changes

sycl/test-e2e/format.py Outdated Show resolved Hide resolved

aelovikov-intel reviewed Jan 21, 2025

View reviewed changes

sycl/test-e2e/format.py Outdated Show resolved Hide resolved

sarnex reviewed Jan 21, 2025

View reviewed changes

ayylol temporarily deployed to WindowsCILock January 21, 2025 22:21 — with GitHub Actions Inactive

Maetveis reviewed Jan 22, 2025

View reviewed changes

sycl/test-e2e/TriStateExpr.py Outdated Show resolved Hide resolved

Maetveis reviewed Jan 22, 2025

View reviewed changes

sycl/test-e2e/format.py Outdated Show resolved Hide resolved

Maetveis reviewed Jan 22, 2025

View reviewed changes

ayylol added 3 commits January 22, 2025 07:09

Address comments related to new expressions

cdd3aa1

Rename triples to build_target where appropriate

0be2b74

Address 2 more comments

5a2b31f

ayylol temporarily deployed to WindowsCILock January 22, 2025 16:22 — with GitHub Actions Inactive

ayylol temporarily deployed to WindowsCILock January 22, 2025 17:36 — with GitHub Actions Inactive

Add more build-mode features

64bccab

ayylol had a problem deploying to WindowsCILock January 24, 2025 16:51 — with GitHub Actions Error

Add any-target-is- features

9159957

ayylol temporarily deployed to WindowsCILock January 24, 2025 17:27 — with GitHub Actions Inactive

ayylol had a problem deploying to WindowsCILock January 24, 2025 18:02 — with GitHub Actions Error

Add missing comma to build-only features list

8f8b648

ayylol temporarily deployed to WindowsCILock January 24, 2025 18:13 — with GitHub Actions Inactive

aelovikov-intel approved these changes Jan 24, 2025

View reviewed changes

ayylol had a problem deploying to WindowsCILock January 24, 2025 19:02 — with GitHub Actions Error

ayylol added 3 commits January 24, 2025 11:58

Format

71ebbe7

Merge branch 'sycl' into tristate-expr-logic

87c0fa5

Revert changes in matrix, bindless_images and syclcompat tests

3e75a1e

ayylol had a problem deploying to WindowsCILock January 24, 2025 20:02 — with GitHub Actions Error

Revert changes to adapters tests

737ab86

ayylol temporarily deployed to WindowsCILock January 24, 2025 20:05 — with GitHub Actions Inactive

ayylol marked this pull request as ready for review January 24, 2025 20:13

ayylol removed request for a team January 24, 2025 20:16

ayylol temporarily deployed to WindowsCILock January 24, 2025 20:37 — with GitHub Actions Inactive

uditagarwal97 approved these changes Jan 24, 2025

View reviewed changes

aelovikov-intel merged commit 8a27954 into intel:sycl Jan 24, 2025
16 checks passed

ayylol deleted the tristate-expr-logic branch January 25, 2025 03:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SYCL][E2E] Add logic to react to `REQUIRED`/`UNSUPPORTED` in `build-only` #16725

[SYCL][E2E] Add logic to react to `REQUIRED`/`UNSUPPORTED` in `build-only` #16725

ayylol commented Jan 21, 2025 •

edited

Loading

ayylol commented Jan 21, 2025

aelovikov-intel Jan 21, 2025

ayylol Jan 22, 2025

sarnex left a comment

sarnex Jan 21, 2025

ayylol Jan 22, 2025

sarnex Jan 22, 2025

ayylol Jan 22, 2025

aelovikov-intel Jan 22, 2025

aelovikov-intel Jan 22, 2025

sarnex Jan 23, 2025

Maetveis Jan 22, 2025 •

edited

Loading

ayylol Jan 22, 2025

Maetveis Jan 22, 2025

aelovikov-intel Jan 22, 2025

aelovikov-intel left a comment

ayylol commented Jan 24, 2025 •

edited

Loading

uditagarwal97 left a comment

[SYCL][E2E] Add logic to react to REQUIRED/UNSUPPORTED in build-only #16725

[SYCL][E2E] Add logic to react to REQUIRED/UNSUPPORTED in build-only #16725

Conversation

ayylol commented Jan 21, 2025 • edited Loading

ayylol commented Jan 21, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sarnex left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Maetveis Jan 22, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aelovikov-intel left a comment

Choose a reason for hiding this comment

ayylol commented Jan 24, 2025 • edited Loading

uditagarwal97 left a comment

Choose a reason for hiding this comment

[SYCL][E2E] Add logic to react to `REQUIRED`/`UNSUPPORTED` in `build-only` #16725

[SYCL][E2E] Add logic to react to `REQUIRED`/`UNSUPPORTED` in `build-only` #16725

ayylol commented Jan 21, 2025 •

edited

Loading

Maetveis Jan 22, 2025 •

edited

Loading

ayylol commented Jan 24, 2025 •

edited

Loading