[REQ] Separate CUDA and CPU compilation configs #1322
Description
Description
Some of Warp's config options currently control both the compilation of CUDA kernels and CPU kernels. We should consider splitting them for more control when they don't quite line up and users may want different behavior for each type of kernel.
Perhaps it could even distinguish between different CUDA devices. For example wp.config.optimization_level["cuda:1"] = 2 when it provides an advantage for that specific GPU.
Context
#1310 illustrates that while initially it appeared desirable compile CPU kernels at optimization level 3 (just like CUDA kernels), it turned out level 2 is a more balanced default. While using config.optimization_level=None can accommodate that by using different effective defaults, it means it's currently not possible to compile CUDA kernels at level 2 while compiling CPU kernels at level 3. Perhaps it can be done by switching the config in between launches, but this would be very fragile.
Note that kernels with modules="unique" could also take overrides for config settings.