Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show failed assertion trace #371

Closed

Conversation

hoangmaihuy
Copy link

/claim #354

Because the tests are macro-expansion from both Tests.apply and assertImpl so the original assertion position is lost after Tracer.apply converts test expressions into Expr[Unit]

val trees: Expr[Seq[AssertEntry[T]]] = Expr.ofSeq(ess.map(e => makeAssertEntry(e, codeOf(e))))

This PR is an attempt to preserve original assertion's position by introducing pos: String field in AssertEntry, which is formatted as ${sourceFile}:${line} so that we can output failed assertion location under AssertionError like:

X test.utest.examples.HelloTests.test4 2ms
[113]   utest.AssertionError:  1 == 2
[113]   at .../utest/utest/test/src/test/utest/examples/HelloTests.scala:21
[113]     utest.asserts.Asserts$.assertImpl(Asserts.scala:31)
[113]     test.utest.examples.HelloTests$.$init$$$anonfun$1$$anonfun$4(HelloTests.scala:5)

@hoangmaihuy hoangmaihuy force-pushed the fix-failed-assertion-trace branch 2 times, most recently from c2b8de2 to 6ba6725 Compare March 18, 2025 15:38
@hoangmaihuy hoangmaihuy force-pushed the fix-failed-assertion-trace branch from 6ba6725 to 493eeb6 Compare March 18, 2025 16:32
@lihaoyi
Copy link
Member

lihaoyi commented Mar 19, 2025

the original assertion position is lost after Tracer.apply converts test expressions into Expr[Unit]

Perhaps this is the thing we should focus on. The current PR that plumbs through the changes as runtime values works, but I don't think it's the right fix here. Why is Tracer.apply losing the assertion positions, and can we fix it?

@hoangmaihuy
Copy link
Author

hoangmaihuy commented Mar 19, 2025

@lihaoyi I made futher debugging and see that Tracer.apply convert test expressions into Seq[AssertEntry[...]] in test file:

val esx: scala.collection.immutable.Seq[utest.asserts.AssertEntry[scala.Boolean]] = utest.asserts.AssertEntry.apply[scala.Boolean]("1 == 2", ((logger: scala.Function1[utest.TestValue, scala.Unit]) => false))

utest.asserts.Asserts.assertImpl(esx: _*)

And we have assertImpl is a function which actually evaluate those AssertEntry, that's why when assertImpl throw exception the stack trace only show

utest.asserts.Asserts$.assertImpl(Asserts.scala:31)
test.utest.examples.HelloTests$.$init$$$anonfun$1$$anonfun$4(HelloTests.scala:5)

because utest.asserts.Asserts.assertImpl(esx: _*) is already inlined into main source code. I think the difference in macro-expansion between scala 2 and 3 introduces this problem.

To fix this issue, assertion error should be thrown in AssertEntry function instead of assertImpl, but I haven't figured out how to implement that because it will affect scala 2 macro as well.

Besides, I think output full path of error location is not a bad choice because I can click on the path to jump into that location in editor/IDE just like scala's compile error.

@lihaoyi
Copy link
Member

lihaoyi commented Mar 22, 2025

As far as the linked ticket is concerned, I won't accept the approach in this PR. It's a decent workaround, but it doesn't really address the root issue: neither fixing it nor investigating and explaining why it is unfixable. As much as I want a fix myself, I'll hold off for a deeper fix or investigation

@lihaoyi lihaoyi closed this Mar 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants