GPU support?

[gary-stanford]

LightGBMLSS is amazing! I'm particularly interested in mixed distributions. It can be slow to train with larger datasets though. Can training be accelerated with a GPU? I have a GPU in my system and the versions of lightgbm and pytorch installed include GPU support but I can't figure out how to make LightGBMLSS use it.

[StatMixedML]

@gary-stanford Thanks for your interest in the project!

To address your concerns:

• LightGBMLSS can indeed leverage GPU acceleration as long as the custom objective functions are supported by the GPU. You can use LightGBMLSS just as you would use LightGBM, applying the same LightGBM parameters for GPU usage. However, it's important to note that the GPU will not speed up the mixture densities calculations. While PyTorch is used within LightGBMLSS, it's primarily for its auto-grad functionality to derive gradients and Hessians; no other GPU-tensor operations are involved, aside from the built-in GPU training capabilities of LightGBM.

• I agree that the mixture of densities tends to be slow, likely due to the categorical distribution used for weighting the densities. I haven't benchmarked TensorFlow Probability's runtime for mixture densities, but it might be more efficient in general. Plus, it offers a broader range of distributions, which could be useful. Maybe I should migrate the project to TFP.