Initial GPU pipeline #202

kurapov-peter · 2024-07-30T15:59:07Z

This adds a GPU pipeline and the necessary glue code for it to work with ocloc through the gpu-module-to-binary.

The patch mainly adds the gen dialect to hold the target attribute for binary generation (has nothing to do with Triton's gen dialect, although sits at the same level, alongside LLVM) as well as some target-specific parameters.

The lowering uses gpu-to-llvm-spv to generate OpenCL calls from GPU operations. The pass expects a SPIR-V target attached with some required settings. It is thus temporarily attached to make the pass happy. Down the pipeline, it is replaced with the gen.target (otherwise the logic of binary generation would produce a spirv as a result). GPUOpsLowering.h is temporarily copy-pasted until kernel signature conversion is available upstream. The upper part of the pipeline is "dumb" - generalizes and converts the input linalg into parallel loops to then map to GPU.

The current implementation serializes the GPU module to a binary SPIR-V via LLVM's SPIR-V backend and uses ocloc to convert that into a GEN binary that is wrapped into a gpu.obj. Ocloc is searched for in the PATH. Although there some code for its discovery in the base toolkit, the current location of the binary is inside vtune there, so it is unclear whether this should be implemented at all.

As is, this does not work with the ocl wrappers (#191). There is no gpux, so the path expects wrappers to follow naming conventions.

AndreyPavlenko · 2024-07-30T22:02:04Z

src/gc-opt/gc-opt.cpp

@@ -46,13 +52,23 @@ int main(int argc, char *argv[]) {
 #endif
  mlir::registerAllPasses();
  mlir::gc::registerCPUPipeline();
+  mlir::gc::registerGPUPipeline();


Suggested change

mlir::gc::registerGPUPipeline();

#ifdef GC_USE_GPU

mlir::gc::registerGPUPipeline();

#endif

What is this for?

We have the build option for enabling/disabling GPU. It could make sense if GPU is disabled.

AndreyPavlenko · 2024-07-30T22:04:52Z

lib/gc/CAPI/Passes.cpp

@@ -18,6 +18,7 @@ using namespace mlir::cpuruntime;

 namespace mlir::gc {
 void registerCPUPipeline();
+void registerGPUPipeline();


Suggested change

void registerGPUPipeline();

#ifdef GC_USE_GPU

void registerGPUPipeline();

#endif

AndreyPavlenko · 2024-07-30T22:05:22Z

lib/gc/CAPI/Passes.cpp

@@ -29,6 +30,7 @@ extern "C" {

 MLIR_CAPI_EXPORTED void mlirRegisterAllGCPassesAndPipelines() {
  registerCPUPipeline();
+  registerGPUPipeline();


Suggested change

registerGPUPipeline();

#ifdef GC_USE_GPU

registerGPUPipeline();

#endif

AndreyPavlenko · 2024-07-30T22:06:00Z

src/gc-opt/gc-opt.cpp

@@ -32,6 +37,7 @@

 namespace mlir::gc {
 void registerCPUPipeline();
+void registerGPUPipeline();


Suggested change

void registerGPUPipeline();

#ifdef GC_USE_GPU

void registerGPUPipeline();

#endif

AndreyPavlenko · 2024-07-30T22:09:14Z

lib/gc/Transforms/Pipeline.cpp

+  PassPipelineRegistration<>("gc-gpu-pipeline",
+                             "The GPU pipeline for Graph Compiler",
+                             populateGPUPipeline);
+}


Suggested change

}

}

#endif

AndreyPavlenko · 2024-07-30T22:09:44Z

lib/gc/Transforms/Pipeline.cpp

@@ -145,10 +147,45 @@ void populateCPUPipeline(mlir::OpPassManager &pm) {
  populateLLVMPasses(pm);
 }

+void populateGPUPipeline(mlir::OpPassManager &pm) {


Suggested change

void populateGPUPipeline(mlir::OpPassManager &pm) {

#ifdef GC_USE_GPU

void populateGPUPipeline(mlir::OpPassManager &pm) {

AndreyPavlenko · 2024-07-30T22:11:29Z

lib/gc/Transforms/Pipeline.cpp

+  pm.addPass(createGpuGenAttachTarget());
+  GpuModuleToBinaryPassOptions gpuModuleToBinaryPassOptions;
+  pm.addPass(createGpuModuleToBinaryPass(gpuModuleToBinaryPassOptions));
+}


Suggested change

}

}

#endif

AndreyPavlenko · 2024-07-30T22:11:38Z

lib/gc/Transforms/Pipeline.cpp

 void registerCPUPipeline() {
  PassPipelineRegistration<>("gc-cpu-pipeline",
                             "The CPU pipeline for Graph Compiler",
                             populateCPUPipeline);
 }

+void registerGPUPipeline() {


Suggested change

void registerGPUPipeline() {

#ifdef GC_USE_GPU

void registerGPUPipeline() {

scripts/compile.sh

lib/gc/Target/LLVM/CMakeLists.txt

AndreyPavlenko · 2024-07-31T02:03:15Z

src/gc-opt/gc-opt.cpp

+  // gpu.module op
+  mlir::registerAllToLLVMIRTranslations(registry);
+  mlir::gen::registerGenTargetInterfaceExternalModels(registry);
+  mlir::registerGENDialectTranslation(registry);
 #ifdef GC_USE_GPU


Probably, it should be renamed to GC_USE_IMEX here and in all other places.

Menooker · 2024-07-31T02:46:46Z

lib/gc/Dialect/LLVMIR/CMakeLists.txt

@@ -0,0 +1,20 @@
+add_mlir_dialect_library(MLIRGENDialect


Should the directory name be "GEN" instead of "LLVMIR"?

No, the dialect is at the same level as llvmir, this follows the upstream structure.

Menooker · 2024-07-31T02:58:00Z

lib/gc/Target/LLVM/GEN/Target.cpp

+}
+
+std::optional<SmallVector<char, 0>>
+GenSerializer::compileToBinary(const std::string &serializedSPV) {


Shall we skip this step and treat the spirv code as the final binary? This can free us from findTool("ocloc") which depends on the environment, and we can pass the compilation issue to OCL runtime.

This can be controlled by the targetOptions.getCompilationTarget(). The binary generation here is for latency elimination on the first execution when the target arch is known (inference).

This reverts commit a31238e.

kurapov-peter · 2024-08-05T11:14:40Z

lib/gc/Target/LLVM/CMakeLists.txt

+  MLIRSupport
+  MLIRGPUDialect
+  MLIRTargetLLVM
+  LLVMSPIRVCodeGen


This is weird, I couldn't reproduce the linking problem locally and I get a runtime problem with static llvm options reinitialization when I include LLVMSPIRVCodeGen as a dependency.

dchigarev · 2024-08-06T16:19:38Z

lib/gc/Transforms/Pipeline.cpp

+  GpuModuleToBinaryPassOptions gpuModuleToBinaryPassOptions;
+  pm.addPass(createGpuModuleToBinaryPass(gpuModuleToBinaryPassOptions));


Which tests did you use to verify the pipeline?

If I try to run the GPU pipeline on this simple matmul test the gpu-module-to-binary pass fails with the following error:

error: LLVM Translation failed for operation: builtin.unrealized_conversion_cast

don't we need to add reconcile-unrealized-casts somewhere?

UPD:
simply adding pm.addPass(createReconcileUnrealizedCastsPass()); before the gpu-to-bin pass didn't help :(

Testing it on a simple vector add for now. I'll add it once we can execute it. Yes, we will need the reconcile pass for the pipeline to be complete. I'd like to get to an end-to-end working scenario first though.

leshikus · 2024-08-12T14:36:17Z

@kurapov-peter What should I change in my mirror branch to ensure GPU is actually accessed?
https://github.com/intel/graph-compiler/actions/runs/10353628700

lmontigny · 2024-08-22T15:02:24Z

Let's review this one before end of iteration 4

lmontigny

Ok, let's have the initial version of the GPU pipeline

kurapov-peter linked an issue Jul 30, 2024 that may be closed by this pull request

Construct initial GPU pipeline #155

Open

kurapov-peter mentioned this pull request Jul 30, 2024

Add ocloc and opencl runtime to CI #203

Closed

AndreyPavlenko reviewed Jul 30, 2024

View reviewed changes

AndreyPavlenko reviewed Jul 31, 2024

View reviewed changes

scripts/compile.sh Outdated Show resolved Hide resolved

AndreyPavlenko reviewed Jul 31, 2024

View reviewed changes

lib/gc/Target/LLVM/CMakeLists.txt Show resolved Hide resolved

AndreyPavlenko reviewed Jul 31, 2024

View reviewed changes

Menooker reviewed Jul 31, 2024

View reviewed changes

kurapov-peter mentioned this pull request Jul 31, 2024

Update llvm-version.txt #204

Merged

kurapov-peter force-pushed the pakurapo/gpu-module-legalize branch from c1d1097 to 1287334 Compare August 2, 2024 13:48

kurapov-peter mentioned this pull request Aug 2, 2024

Rename GC_ENABLE_GPU to GC_ENABLE_IMEX & clean up properties usage #205

Closed

kurapov-peter added 18 commits August 5, 2024 02:50

Add gc-gpu-legalize-module pass

492d525

Add gc-gpu-signatures-to-llvm pass

f8b665e

Add gen dialect to hold the gen target

7f8c611

Add gen target and a lowering pipeline through gpu-module-to-binary

fa3ba4b

Add ocloc integration

59d586b

Add gpu pipeline registration

dbeb704

Fix typo

6ee3d0b

Fix static gc-opt build & add comments for components registration

9291855

Fix warnings

3cbebc1

Disable clang-tidy on the attribute definition

8193d65

Fix licences

643f363

Move xegpu pass to imex-only build

2013270

Fix python CAPI linkage

3f7a71a

fixup! Fix python CAPI linkage

ec17d13

Fix merge issues

ce9e66f

Fix GCExecutionEngineTests linkage

09e8321

Add SPIRVCodeGen dependency to the gen target

c51053c

Revert "Add SPIRVCodeGen dependency to the gen target"

eefddb6

This reverts commit a31238e.

kurapov-peter force-pushed the pakurapo/gpu-module-legalize branch from e876bb7 to eefddb6 Compare August 5, 2024 09:51

Add LLVMSPIRVCodeGen dependency

c2b485e

kurapov-peter added the ready to review label Aug 5, 2024

kurapov-peter commented Aug 5, 2024

View reviewed changes

dchigarev reviewed Aug 6, 2024

View reviewed changes

lmontigny requested review from dchigarev and Menooker August 22, 2024 15:01

lmontigny approved these changes Aug 22, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial GPU pipeline #202

Initial GPU pipeline #202

kurapov-peter commented Jul 30, 2024

AndreyPavlenko Jul 30, 2024

kurapov-peter Jul 31, 2024

AndreyPavlenko Jul 31, 2024

AndreyPavlenko Jul 30, 2024

AndreyPavlenko Jul 30, 2024

AndreyPavlenko Jul 30, 2024

AndreyPavlenko Jul 30, 2024

AndreyPavlenko Jul 30, 2024

AndreyPavlenko Jul 30, 2024

AndreyPavlenko Jul 30, 2024

AndreyPavlenko Jul 31, 2024

Menooker Jul 31, 2024

kurapov-peter Jul 31, 2024

Menooker Jul 31, 2024

kurapov-peter Jul 31, 2024

kurapov-peter Aug 5, 2024

dchigarev Aug 6, 2024 •

edited

Loading

kurapov-peter Aug 7, 2024

leshikus commented Aug 12, 2024 •

edited

Loading

lmontigny commented Aug 22, 2024

lmontigny left a comment

	void populateGPUPipeline(mlir::OpPassManager &pm) {
	#ifdef GC_USE_GPU
	void populateGPUPipeline(mlir::OpPassManager &pm) {

	void registerGPUPipeline() {
	#ifdef GC_USE_GPU
	void registerGPUPipeline() {

		GpuModuleToBinaryPassOptions gpuModuleToBinaryPassOptions;
		pm.addPass(createGpuModuleToBinaryPass(gpuModuleToBinaryPassOptions));

Initial GPU pipeline #202

Are you sure you want to change the base?

Initial GPU pipeline #202

Conversation

kurapov-peter commented Jul 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dchigarev Aug 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leshikus commented Aug 12, 2024 • edited Loading

lmontigny commented Aug 22, 2024

lmontigny left a comment

Choose a reason for hiding this comment

dchigarev Aug 6, 2024 •

edited

Loading

leshikus commented Aug 12, 2024 •

edited

Loading