-
Notifications
You must be signed in to change notification settings - Fork 473
[REQUEST] Support running GPU kernels with arrays in system memory on Grace Hopper #422
Copy link
Copy link
Open
Open
Copy link
Labels
Milestone
Description
Description
It should be possible to mix-and-match arrays on Grace Hopper systems, e.g. CPU kernels can access arrays residing in GPU memory and vice versa. A runtime check requiring all arrays to be allocated on the same device as the kernel is one of the things preventing this from working:
Lines 4945 to 4949 in 211b962
| # check device | |
| if value.device != device: | |
| raise RuntimeError( | |
| f"Error launching kernel '{kernel.key}', trying to launch on device='{device}', but input array for argument '{arg_name}' is on device={value.device}." | |
| ) |
There might be additional changes required for this capability to be performant.
Similarly, it should be possible to mix-and-match GPU arrays when peer access is enabled on peer-capable GPUs.
Context
Take advantage of platform features.
Reactions are currently unavailable