[BUG/Question]: Could not find implicit value for evidence parameter of type upickle.default.ReadWriter[T]

[cbcoutinho]

Tapir: 1.0.0-M1
uPickle: 1.5.0
sttp: 3.5.1

Scala version: 2.12.13

Describe the bug

I am working with a JSON document that omits optional values, yet I can't figure out how to use optional fields with tapir and uPickle.

I would like to generate a client that can accept all to accept all three example responses for Message. Currently the commented baseEndpoint does not compile because Tapir cannot find an implicit value. Otherwise the tests below pass w.r.t uPickle

I'm sure this is an easy question but it's not clear from the example (TapirJsonuPickleTests.scala) how to use a custom encoder/decoder with jsonBody.

How to reproduce?

package com.example

import org.scalatest.flatspec.AnyFlatSpec
import sttp.tapir._
import sttp.tapir.generic.auto._
import sttp.tapir.json.upickle._

object TapirJsonuPickleCodec extends TapirJsonuPickle

object OptionPickler extends upickle.AttributeTagged {
  override implicit def OptionWriter[T: Writer]: Writer[Option[T]] =
    implicitly[Writer[T]].comap[Option[T]] {
      case Some(x) => x
      case None    => null.asInstanceOf[T]
    }

  override implicit def OptionReader[T: Reader]: Reader[Option[T]] =
    new Reader.Delegate[Any, Option[T]](implicitly[Reader[T]].map(Some(_))) {
      override def visitNull(i: Int) = None
    }
}

class TapirJsonSpec extends AnyFlatSpec {

  import OptionPickler.{ ReadWriter, macroRW, read }

  case class Message(a: String, b: Option[String] = None)
  object Message {
    implicit val rw: ReadWriter[Message] = macroRW
  }

  // What does a messageDecoder have to do with `jsonBody`?
  // val messageDecoder: JsonCodec[Message] = TapirJsonuPickleCodec.readWriterCodec[Message]

  // val baseEndpoint: PublicEndpoint[String, String, Message, Any] = endpoint
  // .in(path[String]("id"))
  // .errorOut(stringBody)
  // .out(jsonBody[Message])

  it should "parse json" in {
    val str = """{"a": "1", "b": "2"}"""
    read[Message](str)
  }

  it should "handle nulls" in {
    val str = """{"a": "1", "b": null}"""
    read[Message](str)
  }

  it should "handle missing values" in {
    val str = """{"a": "1"}"""
    read[Message](str)
  }
}

[cbcoutinho]

Quick update, after some further searching it appears to be possible to use custom encoders with jsonBody by just re-implementing the method for the custom ReadWriter in the root case class from the response, as follows:

package com.example

import scala.util.Failure
import scala.util.Success
import scala.util.Try

import org.scalatest.flatspec.AnyFlatSpec
import sttp.tapir.Codec.JsonCodec
import sttp.tapir._
import sttp.tapir.generic.auto._

object OptionPickler extends upickle.AttributeTagged {
  override implicit def OptionWriter[T: Writer]: Writer[Option[T]] =
    implicitly[Writer[T]].comap[Option[T]] {
      case Some(x) => x
      case None    => null.asInstanceOf[T]
    }

  override implicit def OptionReader[T: Reader]: Reader[Option[T]] =
    new Reader.Delegate[Any, Option[T]](implicitly[Reader[T]].map(Some(_))) {
      override def visitNull(i: Int) = None
    }
}

class TapirJsonSpec extends AnyFlatSpec {

  import OptionPickler._

  case class Message(a: String, b: Option[String] = None)
  object Message {
    implicit val rw: ReadWriter[Message] = macroRW

    def jsonBody[T: ReadWriter: Schema]: EndpointIO.Body[String, T] =
      anyFromUtf8StringBody(
        readWriterCodec[T]
      )

    implicit def readWriterCodec[T: ReadWriter: Schema]: JsonCodec[T] =
      Codec.json[T] { s =>
        Try(read[T](s)) match {
          case Success(v) => DecodeResult.Value(v)
          case Failure(e) =>
            DecodeResult.Error(s, DecodeResult.Error.JsonDecodeException(errors = List.empty, e))
        }
      }(t => write(t))
  }

  import Message._

  val baseEndpoint: PublicEndpoint[String, String, Message, Any] = endpoint
    .in(path[String]("id"))
    .errorOut(stringBody)
    .out(jsonBody[Message])

  it should "parse json" in {
    val str = """{"a": "1", "b": "2"}"""
    read[Message](str)
  }

  it should "handle nulls" in {
    val str = """{"a": "1", "b": null}"""
    read[Message](str)
  }

  it should "handle missing values" in {
    val str = """{"a": "1"}"""
    read[Message](str)
  }
}

[adamw]

Hm I've never used uPickle so I afraid I won't help much, but as you can see the integration layer is very thin; in essence you just need to provide a two-way translation between the json and the parsed representation:

trait TapirJsonuPickle {

  def jsonBody[T: ReadWriter: Schema]: EndpointIO.Body[String, T] = anyFromUtf8StringBody(readWriterCodec[T])

  implicit def readWriterCodec[T: ReadWriter: Schema]: JsonCodec[T] =
    Codec.json[T] { s =>
      Try(read[T](s)) match {
        case Success(v) => Value(v)
        case Failure(e) => Error(s, JsonDecodeException(errors = List.empty, e))
      }
    } { t => write(t) }
}

Here that's done basing on an implicit ReadWriter from uPickle. Are we missing some additional handling of optional values? I assumed that this is all done as part of creating the ReadWriter instance?

[cbcoutinho]

From my understanding of the uPickle docs, the ReadWriter referenced in TapirJsonuPickle is the default one implemented for all primitive types (upickle.default.ReadWriter). The Option[A] SerDe is provided by the custom pickler OptionPickler, which provides its own ReadWriter (subclass?) instance.

I'm a bit new to the three-type function signature used in these methods, could you point me to a reference where I can learn what it is supposed to represent? I suspect the OptionPickler.ReadWriter instance to be a subclass of the default ReadWriter. Would type bounds be applicable here?

[adamw]

Hm I'm not sure I understand. What do you mean by three-type function signature used in these methods? :) Is it the T: ReadWriter: Schema? If so, this notation is a shorthand for:

 implicit def readWriterCodec[T](implicit x: ReadWriter[T], implicit y: Schema[T]) JsonCodec[T]

that is, there must be a ReadWriter and Schema instance in the implicit scope defined for the type T

[cbcoutinho]

Yes precisely, that's what I meant. In the meantime I read that these are called context bounds. Ok, then the problem appears to be these scopes. Is there a way to require the existence of any ReadWriter by requiring a subclass of AttributeTagged?

The upickle.default package appears to be an empty implementation of AttributeTagged, whereas the OptionPickler object above overrides some internal methods, resulting in a new ReadWriter.

I think the crux is that the [T: ReadWriter: Schema] type signature used by by TapirJsonuPickle does not enforce the subclassing of AttributeTagged, rather it just requires the default ReadWriter be in implicit scope.

[adamw]

The signature requires that any ReadWriter is in the implicit scope. You can provide a custom one by defining e.g. a custom implicit val.

Do you have the code to serialise an object to json using upickle (without tapir for the moment)? I think that might be a good start.

[cbcoutinho]

My previous comment includes code that imnplements the custom ReadWriter; however, it requires re-implementing the TapirJsonPickle methods. This is a fine workaround considering the amount of code involved. I've since migrated to circe so I won't be investigating this further.

Thanks for the follow-up @adamw!