Opaque types may cause deserialization to never complete

[cchepelov]

Using zio-json 2.0.0-M1 under Scala 3.0.2.

The following scastie worksheet works fine: https://scastie.scala-lang.org/VGtUGBteQjiB9Y8Xlz4K5A

for reference, the code contents is:

import zio.json.*

opaque type Email = String
object Email:
  def apply(value: String): Email = value
  given JsonDecoder[Email] = JsonDecoder[String].map(Email.apply)  

extension (x: Email)
  def value: String = x



case class Structure(name: String, 
                     email: Email)
object Structure {
  given JsonDecoder[Structure] = DeriveJsonDecoder.gen 
}

val json = """{ "name": "Toto", "email": "[email protected]"}"""

val data = json.fromJson[Structure]

println(data)

However, splitting the above code into three separate files (as one would usually do) causes the fromJson call to apparently never return:

hello/Email.scala:

package hello

import zio.json.JsonDecoder

opaque type Email = String
object Email:
  def apply(value: String): Email = value

  given JsonDecoder[Email] = JsonDecoder[String].map(Email.apply)

extension (x: Email)
  def value: String = x

hello/Structure.scala:

package hello

import zio.json.{DeriveJsonDecoder, JsonDecoder}

case class Structure(name: String,
                     email: Email)   // Change this to 'String' 
object Structure {
  given JsonDecoder[Structure] = DeriveJsonDecoder.gen
}

hello/TestCase.scala:

package hello

import zio.json.*

object TestCase extends App  {
  val json = """{ "name": "Toto", "email": "[email protected]"}"""

  val dataOrElse = json.fromJson[Structure]

  dataOrElse match {
    case Right(data) =>
      println(data)
      println(data.email)
      println(data.email: String)
      println(data.email.value) // extension method
    case Left(error) =>
      println(s"bummer, ${error}")
  }

}

Expected behaviour
calling """{ "name": "Toto", "email": "[email protected]"}""".fromJson[Structure] works even if the "email" field is defined as an opaque field with a suitable decoder

Observed behaviour
While the code appears to work as long as everything is in the same compilation unit (as it is in Scastie), it fails when split over distinct files.

Changing the Structure's email field type back to String works around the issue.

[cchepelov]

The relevant stack trace is following. Would it suggest a bad interaction between magnolia and scala's internal implementation of givens?

This would suggest #434 result would be dearly needed

"run-main-0" #142 prio=5 os_prio=0 tid=0x000002394e072800 nid=0x4c8c in Object.wait() [0x0000008fbd7fd000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Unknown Source)
        at scala.runtime.LazyVals$.wait4Notification(LazyVals.scala:89)
        - locked <0x00000000faf7d980> (a java.lang.Object)
        at hello.Email$package$Email$.given_JsonDecoder_Email(Email.scala:9)
        at hello.Email$package$Email$.given_JsonDecoder_Email(Email.scala:9)
        at hello.Structure$.$anonfun$6(Structure.scala:8)
        at hello.Structure$$$Lambda$4540/2065787641.apply(Unknown Source)
        at magnolia.CallByNeed$.apply$$anonfun$1(interface.scala:106)
        at magnolia.CallByNeed$$$Lambda$4535/1507160179.apply(Unknown Source)
        at magnolia.CallByNeed.value(interface.scala:110)
        at magnolia.CaseClass$$anon$2.typeclass(interface.scala:28)
        at zio.json.DeriveJsonDecoder$.zio$json$DeriveJsonDecoder$$anon$3$$_$tcs$$anonfun$1(macros.scala:112)
        at zio.json.DeriveJsonDecoder$$anon$3$$Lambda$4592/995154638.apply(Unknown Source)
        at scala.collection.ArrayOps$.map$extension(ArrayOps.scala:924)
        at scala.IArray$package$IArray$.map(IArray.scala:179)
        at zio.json.DeriveJsonDecoder$$anon$3.tcs(macros.scala:112)
        at zio.json.DeriveJsonDecoder$$anon$3.unsafeDecode(macros.scala:137)
        at zio.json.JsonDecoder.decodeJson(decoder.scala:79)
        at zio.json.JsonDecoder.decodeJson$(decoder.scala:44)
        at zio.json.DeriveJsonDecoder$$anon$3.decodeJson(macros.scala:96)
        at zio.json.package$DecoderOps$.fromJson$extension(package.scala:28)
        at hello.TestCase$.<clinit>(TestCase.scala:8)
        at hello.TestCase.main(TestCase.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

[cchepelov]

Additionally:

declaring Email's JsonDecoder this way:

  inline given JsonDecoder[Email] = JsonDecoder[String].map(Email.apply)

will cause the generated decoder to immediately crash due to NoSuchMethodError:

[error] (run-main-0) java.lang.NoSuchMethodError: hello.Email$package$Email$.given_JsonDecoder_Email()Lzio/json/JsonDecoder;
[error] java.lang.NoSuchMethodError: hello.Email$package$Email$.given_JsonDecoder_Email()Lzio/json/JsonDecoder;
[error]    at hello.Structure$.$anonfun$6(Structure.scala:8)
[error]    at magnolia.CallByNeed$.apply$$anonfun$1(interface.scala:106)
[error]    at magnolia.CallByNeed.value(interface.scala:110)
[error]    at magnolia.CaseClass$$anon$2.typeclass(interface.scala:28)
[error]    at zio.json.DeriveJsonDecoder$.zio$json$DeriveJsonDecoder$$anon$3$$_$tcs$$anonfun$1(macros.scala:112)
[error]    at scala.collection.ArrayOps$.map$extension(ArrayOps.scala:924)
[error]    at scala.IArray$package$IArray$.map(IArray.scala:179)
[error]    at zio.json.DeriveJsonDecoder$$anon$3.tcs(macros.scala:112)
[error]    at zio.json.DeriveJsonDecoder$$anon$3.unsafeDecode(macros.scala:137)
[error]    at zio.json.JsonDecoder.decodeJson(decoder.scala:79)
[error]    at zio.json.JsonDecoder.decodeJson$(decoder.scala:44)
[error]    at zio.json.DeriveJsonDecoder$$anon$3.decodeJson(macros.scala:96)
[error]    at zio.json.package$DecoderOps$.fromJson$extension(package.scala:28)
[error]    at hello.TestCase$.<clinit>(TestCase.scala:8)

(in other words, Magnolia will not notice that the JsonDecoder[Email] is declared inline and has no existence within the jar. Hopefully the native derivation-based generator will be able to pull the decoder AST directly, whether inline or not)

[cchepelov]

zio-json-tc-444.tar.gz

[fsvehla]

Thanks for the report.

[heaven-born]

I have the same issue with zio-json 0.5.0.

Is there a workaround that works with opaque types?

The compiler warns me of the infinite loop when I do this with opaque type: given JsonDecoder[NodeTitle] = JsonDecoder[NodeTitle] or given JsonDecoder[NodeTitle] = JsonDecoder[String].map(NodeTitle.apply)

[dacr]

Same for me with 0.5.0:

[warn] Infinite loop in function body
[warn] zio.json.JsonDecoder.apply[String](this.given_JsonDecoder_UserName).map[String](
[warn]   {
[warn]     def $anonfun(value: String): String = this.apply(value)
[warn]     closure($anonfun)
[warn]   }
[warn] )
[warn]   given JsonDecoder[UserName] = JsonDecoder[String].map(UserName.apply)
[warn]                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

for

opaque type UserName = String
object UserName {
  def apply(value: String): UserName = value

  given JsonDecoder[UserName] = JsonDecoder[String].map(UserName.apply)
}

[PawelJ-PL]

I think it's not related to this library (zio-json), but to the way opaque types works. In this scope, both String and UserName refer to the same type, so trying to retrieve a given instance causes endless loop.

As a workaround you can use

given JsonDecoder[UserName] = JsonDecoder.string

[dacr]

You're right ! And thank you for your feedback :) The example I've just written to test this workaround :

// ---------------------
//> using scala  "3.2.2"
//> using dep "dev.zio::zio:2.0.13"
//> using dep "fr.janalyse::zio-worksheet:2.0.13.0"
//> using dep "dev.zio::zio-json:0.5.0"
//> using options "-Yretain-trees" // When case classes are using default values
// ---------------------

import zio.*, zio.json.*, zio.worksheet.*
import java.time.OffsetDateTime

opaque type UserName = String
object UserName {
  def apply(value: String): UserName = value
  given JsonCodec[UserName]          = JsonCodec.string
}

opaque type LastSeenDateTime = OffsetDateTime
object LastSeenDateTime {
  def apply(value: OffsetDateTime): LastSeenDateTime = value
  given JsonCodec[LastSeenDateTime]                  = JsonCodec.offsetDateTime
}

case class User(
  userName: UserName,
  lastSeen: LastSeenDateTime
) derives JsonCodec

val json =
  """{
    |  "userName":"joe",
    |  "lastSeen":"2021-04-09T17:19:17.000Z"
    |}""".stripMargin

val app =
  for
    person <- ZIO.fromEither(json.fromJson[User])
    _      <- Console.printLine(person.toJsonPretty)
  yield ()

app.unsafeRun