[BUG] Scala3 Schema.derived with recursive structures causes Deadlock with given keyword

[hamnis]

Tapir version: 1.2.8

Scala version: 3.2.2

Describe the bug

When using Schema.derived on a recursive data structure, a deadlock is caused runtime.
It compiles fine, but fails runtime.

How to reproduce?

Minimized example:
https://gist.github.com/hamnis/eee66ef4189cca09b08505bb00d5406a

Additional information

Workaround, use implicit def for this

[adamw]

Yes, this is a known problem: https://tapir.softwaremill.com/en/latest/endpoint/schemas.html#derivation-for-recursive-types-in-scala3

Unfortunately I didn't write in the docs why this is the case. So probably worth investigating and properly documenting what is wrong.

[susliko]

Spent a few hours debugging a stalled app today.

Source of the problem

Code like this:

case class Animal(
  cub: Option[Animal]
) derives Schema

blocks a thread during initialization on java.util.concurrent.CountDownLatch.await.

That latch is a used by Scala lazy val initialization mechanism to ensure some safety.

The blocking on the latch happens due to this line

tapir/core/src/main/scala-3/sttp/tapir/generic/auto/SchemaMagnoliaDerivation.scala

Line 41 in 5cd98a0

val pSchema = enrichSchema(p.typeclass, annotations)

Under the hood p.typeclass tries to reenter this same derivation process when p is Option[Animal]. There is a "cache" to avoid such loops, but it's not used the typeclass search for case class parameters.

What could be done

1. Having the Mirror instance we can get a full qualified type names of all case class parameters. Analyze these names and maybe hard-code cases for Option/List/etc.
2. Write derivation in pure macros instead of Magnolia

@adamw wdyt?

[adamw]

Yeah, unfortunately that's a "known issue" (hence the documentation entry above).

The best quick solution would be to solve this at the level of Magnolia - in a generic way, not specialised to Option/List.

I did try to derive schemas directly once but it proved more challenging that I though initially - at some point I got to rewriting Magnolia, which seemed to miss the point ;). But it might worthwhile revisiting the problem at some point, especially that Scala 3 derivation/macros matured somewhat.