Skip to content

[REQUEST] Support running GPU kernels with arrays in system memory on Grace Hopper #422

@shi-eric

Description

@shi-eric

Description

It should be possible to mix-and-match arrays on Grace Hopper systems, e.g. CPU kernels can access arrays residing in GPU memory and vice versa. A runtime check requiring all arrays to be allocated on the same device as the kernel is one of the things preventing this from working:

warp/warp/context.py

Lines 4945 to 4949 in 211b962

# check device
if value.device != device:
raise RuntimeError(
f"Error launching kernel '{kernel.key}', trying to launch on device='{device}', but input array for argument '{arg_name}' is on device={value.device}."
)

There might be additional changes required for this capability to be performant.

Similarly, it should be possible to mix-and-match GPU arrays when peer access is enabled on peer-capable GPUs.

Context

Take advantage of platform features.

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions