ConsoleLauncher should output diff of expected and actual String values

[carolosf]

As a developer if I am running JUnit tests using the Console Launcher on the command line if two large strings are found to be different and I get "expected:" "but was:" if I want to see what was different I need to copy the strings and manually diff them.

It would be easier if there was --details flag or another flag (I think another flag is a better option) that would show me a diff between the two strings for actionable results. In most cases using an IDE is suitable but in workflows such as Bazel and CI it would be useful if a diff would be displayed.

Apologies if this is already supported didn't find anything obvious in the docs.

[marcphilipp]

Potential library to implement this: https://github.com/java-diff-utils/java-diff-utils

@carolosf We've briefly discussed this in our team call today and think this would be feasible since AssertionFailedError from opentest4j carries actual and expected in separate fields.

We'll have to take a closer look first to decide whether --details diff would be the right way of enabling this. Maybe it could even be enabled by default.

[carolosf]

@marcphilipp

If it becomes the default behaviour it might break IDEs and plug-ins - which is why I suggested a totally separate flag.

I didn't spend much time on it but I noticed previously that the Bazel plugin for IntelliJ does a regex for "expected but was" and splits the results.

Thanks for discussing though - if implemented it would significantly reduce my debugging time!

[marcphilipp]

@carolosf IDEs and build tools usually do not run tests via ConsoleLauncher but use the Launcher API directly so this shouldn't be a problem.

[marcphilipp]

Team decision: Change ConsoleLauncher output for test failures to include diff if actual and expected of AssertionFailedError are of type CharSequence.

[marcphilipp]

@carolosf Would you be interested to work on a PR for this?