Support Kotlin 2.4+ in rewrite-kotlin (recipe DSL compiler plugin pinned to 2.3.20)

[timtebeek]

What version of OpenRewrite are you using?

Latest (rewrite-kotlin currently built against Kotlin 2.3.20).

How are you running OpenRewrite?

As a recipe-authoring consumer via the kotlin-recipe-starter, which puts rewrite-kotlin on kotlinCompilerPluginClasspath to use the Kotlin recipe DSL.

Problem

rewrite-kotlin is pinned to the Kotlin 2.3.x compiler, so consumers cannot move their own kotlin("jvm") Gradle plugin to Kotlin 2.4.x. When they do, compilation of their recipe module fails during :compileKotlin:

e: java.lang.ClassCastException:
   org.jetbrains.kotlin.fir.extensions.FirExtensionRegistrarAdapter$Companion
   cannot be cast to org.jetbrains.kotlin.extensions.ProjectExtensionDescriptor
> Task :compileKotlin FAILED
   > Internal compiler error. See log for more details

• Reproduced by the Dependabot bump in moderneinc/kotlin-recipe-starter#2 (Kotlin 2.3.20 → 2.4.10); CI: https://github.com/moderneinc/kotlin-recipe-starter/actions/runs/29453236383/job/87480488714

Root cause

The Kotlin recipe DSL is a K2 compiler plugin shipped inside rewrite-kotlin
(RecipeCompilerPluginRegistrar → FirExtensionRegistrarAdapter.registerExtension(...)), compiled against kotlin-compiler-embeddable:2.3.20 (rewrite-kotlin/build.gradle.kts, val kotlinVersion = "2.3.20"). The @ExperimentalCompilerApi FIR extension-registration internals are not binary-stable across Kotlin minor versions, so a plugin built for 2.3 cannot be loaded by the 2.4 compiler. This hard-couples a consumer's kotlin("jvm") version to the Kotlin version rewrite-kotlin was built against.

Requested change

Support Kotlin 2.4+ in rewrite-kotlin. Expected work:

1. Bump kotlinVersion (kotlin-compiler-embeddable/reflect/stdlib + kotlin-bom) in rewrite-kotlin/build.gradle.kts to 2.4.x.
2. Bump the kotlin("jvm") plugin used to build rewrite-kotlin itself.
3. Update the lockstep kotlin-bom pins in rewrite-gradle/build.gradle.kts and rewrite-maven/build.gradle.kts (both currently 2.3.20).
4. Fix compile breaks from Kotlin 2.3 → 2.4 FIR/IR/compiler-plugin API changes in the parser (KotlinParser, ScriptCompilerPlugin) and the recipe DSL plugin (RecipeCompilerPluginRegistrar, RecipeFirExtensionRegistrar, RecipeIrGenerationExtension).
5. Publish and align rewrite-recipe-bom so consumers can move to Kotlin 2.4.x.

Are you interested in contributing a fix to OpenRewrite?

Possibly.