WA for xpu Generator to support torch API

[abhilash1910]

@rogerxfeng8 @delock @guoyejun Please review.
UT pass on borealis .

[delock]

What is the original caller looks like? This API looks like an XPU specific one and will make model use this function not compatible with other HW than XPU. What is CUDA code and CPU code should look like if this API is implemented for CUDA and CPU?

[abhilash1910]

The original caller is torch.Generator() which is specific for cpu and for cuda like : torch.cuda.Generator(device=0 ) .[device is optional parameter].
Yes this API is typically for xpu specific ; there is a plan to support in stock pytorch with Jinghui.
This can be referred to here : https://github.com/intel-innersource/frameworks.ai.pytorch.ipex-gpu/blob/72b4b33a87b926748e54f548581796c8544d4548/intel_extension_for_pytorch/xpu/generator.py

[abhilash1910]

• @gujinghui

[delock]

Can you post a link to code where this API is called? @abhilash1910

The original caller is torch.Generator() which is specific for cpu and for cuda like : torch.cuda.Generator(device=0 ) .[device is optional parameter]. Yes this API is typically for xpu specific ; there is a plan to support in stock pytorch with Jinghui. This can be referred to here : https://github.com/intel-innersource/frameworks.ai.pytorch.ipex-gpu/blob/72b4b33a87b926748e54f548581796c8544d4548/intel_extension_for_pytorch/xpu/generator.py

[abhilash1910]

@delock This is for Genslm : https://jira.devtools.intel.com/browse/PYTORCHDGQ-2390 (links are present there).

[delock]

@abhilash1910 I'm not sure whether you refer to the genslm code or pytorch code. I can't see xpu_geenrator in the first link and can only see torch.generator in second link.
https://github.com/intel-sandbox/frameworks.ai.genslm/blob/42eb2bb0eee455e4b94b5db1d7fbacd4f5c505eb/genslm/model.py#L141)

https://github.com/intel-innersource/frameworks.ai.pytorch.private-gpu/blob/b61461f5b1683f8179c92c2a0c0660e9ec63a915/torch/utils/data/distributed.py#L100

[abhilash1910]

Yes @delock for Genslm, this is currently bypassed as to not enable shuffle, but testing on local indicates for enabling shuffle , torch.xpu.Generator() is required. This is through the comment line 148 in link 1 ; I sent the source where torch.Generator() gets called for shuffling ; since torch-gpu will get updated , for now to support in Genslm model code.
Also I think that from xpu perspective, there should be an analogous Generator function .