encodeToJsonElement on an inline class throws "No tag in stack for requested element"

[bartvanheukelom]

Describe the bug

@Serializable @JvmInline
value class Name(val v: String)

Encoding a value of that class to a JSON string works as expected, but encodeToJsonElement throws the exception in the title.

This only occurs when serializing the value directly. It works as expected when embedded in a data class.

It can be worked around by creating a custom serializer that effectively does the same thing as the generated one would.

To Reproduce
For a unit test that reproduces it and shows various workarounds that do work, see https://github.com/bartvanheukelom/jbali/blob/java17/src/jvmTest/java/org/jbali/kotser/TaggedEncoderBugTest.kt

Excepted Behaviour

1. For it to, well, work :D
2. Generally speaking, I've seen various situations over the years where encoding something to a string worked but to an element didn't. This surprises me, since I would expect these 2 encoders to have like 80% of their implementation in common, where the differences in the 20% wouldn't cause bugs such as these. But that is just an outsider's perspective, so please correct me if I'm wrong! I'm not complaining, this is after all a great and free library, but I am curious what's up.

Environment

• Kotlin version: 1.6.0
• Library version: 1.3.0 & 1.3.1
• Kotlin platforms: JVM
• Gradle version: 7.2

[sandwwraith]

Actually, their implementation is very different, because the string is decoded/encoded one-by-one token, while JsonElement is encoded via an internal map structure. That's also the source of the bug — top-level value classes simply can't provide the key for this map. It also wasn't working for primitives but was fixed (#465) before value classes landed in serialization

[bartvanheukelom]

Thanks for your reaction and for quickly fixing this specific bug. Just to be annoying, I've looked at that commit and came up with some more cases where I think this error would still occur. Please see TaggedEncoderUndescriptiveTest in https://github.com/bartvanheukelom/jbali/blob/java17/src/jvmTest/java/org/jbali/kotser/TaggedEncoderBugTest.kt
(this also relates to #1775 / #1768)

As to the JsonEncoder implementation, I understand that it's different and that's what causes these bugs. In my original comment I actually meant to say that, purely seen from the outside (as if it didn't exist yet), I would think they should have 80% in common, since that seems the most practical.

Consider that something like this does work:

inline fun <reified T> Json.encodeToJsonElementAlt(obj): JsonElement =
    decodeFromString(encodeToString(obj))

could JsonTreeEncoder (if that's the right one) not just do something like that, without the extra steps?

[sandwwraith]

So you're telling about the case where we provide a custom serializer with CONTEXTUAL descriptor that sometimes calls plain encodeString()? This is not quite right according to serial kinds contract and can be workarounded by encodeSerializableElement(String.serializer(), ...) (however it may be indeed a performance hit in case of e.g. Int)

I think this is a bit out of the scope of this issue and should be considered in #1775

[bartvanheukelom]

It's two cases:

• testUndescriptiveSer: the custom CONTEXTUAL serializer that calls encodeString, and yeah I figured it violated some contract, but also that with Revisit a buildSerialDescriptor #1775 in mind, it may still be relevant
• testContextual: here ContextualSerializer is used completely by the book (right?), so it should work

Update: actually the problem in case 1 is not the direct use of encodeString. That's my mistake, I actually intended to use encodeSerializableValue(String.serializer().. , and now that I have changed it to do that, it still presents the same problem (as expected, since it basically does the same thing as ContextualSerializer).

The underlying real issue is that any CONTEXTUAL serializer will not be able to properly answer needTopLevelTag without also being given the actual value that should be encoded. But since it's only called by encodeSerializableValue, that looks like an easy fix.

[bartvanheukelom]

I also added a test for decodeFromJsonElement and, somewhat to my surprise, it Just Worked™. So I checked out:

internal fun <T> Json.readJson(element: JsonElement, deserializer: DeserializationStrategy<T>): T {
    val input = when (element) {
        is JsonObject -> JsonTreeDecoder(this, element)
        is JsonArray -> JsonTreeListDecoder(this, element)
        is JsonLiteral, JsonNull -> JsonPrimitiveDecoder(this, element as JsonPrimitive)
    }
    return input.decodeSerializableValue(deserializer)
}

and it made sense. Then I was kind of surprised that the write side isn't "mirrorred" but instead JsonTreeEncoder chooses when to delegate to JsonPrimitiveEncoder.

internal fun <T> Json.writeJson(value: T, serializer: SerializationStrategy<T>): JsonElement {
    lateinit var result: JsonElement
    val encoder = JsonTreeEncoder(this) { result = it }
    encoder.encodeSerializableValue(serializer, value)
    return result
}

But I don't know enough about the internals to give any value judgement of that :)

[sandwwraith]

I see, top-level ContextualSerializer should also be considered

[sandwwraith]

Hm, it seems that it should work without additional handling, because first contextual serializer resolves actual serializer, and only after the encodeSerializableValue with needTopLevelTag is called. But it's still possible to add special handling to be more future-proof

[bubenheimer]

I seem to be having the same type of problem, with a value class wrapping List. Wrapping ImmutableList instead does not help, either. Default generated serializer. Serialization V1.3.3 and V1.4.0-RC, Kotlin 1.7.10. Thoughts?

[bubenheimer]

Can this issue be reopened, please? The List-wrapping inline class scenario is trivial to reproduce via Json serialization (encodeToJsonElement), e.g. using:

@Serializable
@JvmInline
value class Exploring(val value: List<Int>)