[WIP] Use junit platform runner to run junit 3, junit 4 and TestNG tests via vintage engine and testng engine

[olamy]

Overview / Goal

• The PR intends to switch Surefire to use the JUnit Platform runner (i.e. the JUnit 5 infrastructure) to run JUnit 4 tests via the Vintage engine, and also support TestNG under that runner.
• The main changes are removal support for dependency to JUnit 3, for versions of JUnit 4 older than 4.12 and testng older than 6.14.3
• main advantage will be to have only a single implementation to maintain as listener/reporter/launcher of test running platform. Without such change there 5 implementations to maintain which makes any changes extra work which doesn't help to get new contributors to help.

Major Changes

Change Description	Impact
Drop JUnit 3 provider	The PR removes the “JUnit 3 provider” entirely. Junit3 code will be executed via junit vintage only need to upgrade junit dependency to 4.12 at least
Enforce minimum JUnit 4 version	JUnit 4.12 becomes the minimum supported version. Versions earlier than 4.12 are no longer supported.
Route JUnit 4 tests via JUnit Platform + Vintage engine	Instead of using a separate JUnit 4 runner / provider, JUnit 4 tests will be executed via the JUnit Platform infrastructure using the Vintage engine.
Integrate TestNG under JUnit Platform runner	TestNG will now also be run via JUnit Platform, with mappings of TestNG configuration (groups, etc.) to the JUnit Platform infrastructure.\
Code removal / cleanup	A lot of internal code for the old JUnit 3/4/47/TestNG providers is removed for simplification
Version bump / snapshot change	The version is bumped to something like 3.6.0-SNAPSHOT in the PR.
Minimum TestNG version	The PR also sets a minimum supported TestNG version (6.14.3)
Adaptation in existing features	Some features are reworked or adapted (e.g. parallel execution, mapping of “groups” in TestNG to tags/filters, error handling in logs) so they work properly under the new runner approach.

There may be compatibility issues or subtle behavior changes when running JUnit 4 or TestNG tests under the new infrastructure (e.g. differences in how filters, groups, or lifecycle behaviors are handled).

The change consolidates execution under the JUnit Platform, which could simplify the runner architecture

[desruisseaux]

@Tibor17 : it has been reminded a few times that peoples who need JUnit 3 can stay on old versions of Surefire plugin. I have not seen a counter-argument to that. I have not seen neither a counter-argument to the fact that supporting JUnit 3 is an impediment to the development of new features. To recap:

• Why users who want to stay on old JUnit versions could not stay also on old Surefire versions?
• Considering that removing JUnit 3 support is necessary for making improvements easier, why supporting JUnit 3 in latest Surefire is more important than allowing the development of new features (multi-release support, etc.)?

[olamy]

Please note Junit3 code can be still executed but need the junit 4.12 dependency at least

[norrisjeremy]

Just to point out: according to the JUnit 6 release notes, the JUnit Vintage engine is deprecated and (to quote) "it should only be used temporarily while migrating tests to JUnit Jupiter or another testing framework with native JUnit Platform support".

What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?

[olamy]

Just to point out: according to the JUnit 6 release notes, the JUnit Vintage engine is deprecated and (to quote) "it should only be used temporarily while migrating tests to JUnit Jupiter or another testing framework with native JUnit Platform support".

What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?

You will have a few ways to go about this:

• Option 1: Easiest one, just stick with the last Surefire version before this PR is merged by setting it in your pluginManagement. Since you’re already on the older JUnit 3/4 (which are deprecated anyway), it’s totally fine to stay on an older Surefire for now.

• Option 2: Surefire will pick up whatever JUnit version or platform it finds (JUnit, JUnit Platform, or JUnit Vintage) from your pom. So unless you’re moving to JUnit 6.x (or a Vintage version that dropped JUnit 3/4 support), you should be all good.

• Option 3: If you’re thinking about updating things, you could use a tool like OpenRewrite or even an AI helper to migrate your tests to JUnit 5 or 6.

[sormuras]

What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?

It's deprecated, but not for removal. Not slowly, not abruptly, not "everly".

We think of it more as we do of JUnit 3 and 4: they are here to stay. Until the last user vanishes. So does the glue-code in junit-vintage-engine. As long as the JUnit Platform is supported by tools (including Maven), you get all the integration with your testing framework of choice. Through an implementation of a test engine. And in case of JUnit 3 and 4, both projects are in an end-of-life status, no change is to be expected on their side.

Not discussed with the team, yet, but I could think of moving the junit-vintage-engine module as sibling right next to the testng-engine project, making it a standalone top-level project.