[SPARK-51806][BUILD] Upgrade kryo-shaded to 4.0.3
#50586
Conversation
| <artifactId>kryo-shaded</artifactId> | ||
| <version>${kryo.version}</version> | ||
| </dependency> | ||
| <dependency> |
There was a problem hiding this comment.
Kryo 4.0.x depends on version objenesis 2.5.1 , while Spark currently depends on objenesis 3.3. Here, it is uniformly defined as version 3.3.
There was a problem hiding this comment.
I don't think it's necessary to use exclusion. But just to be on the safe side, let me exclude it
|
cc @yaooqinn Let's test 4.0.3 first |
LuciferYang
changed the title
kryo-shaded to 4.0.3
|
will update bench result later |
| basicTypes: Double with unsafe:false 222 225 2 4.5 221.9 0.7X | ||
| Array: Int with unsafe:false 13 14 1 78.0 12.8 12.9X | ||
| Array: Long with unsafe:false 21 21 1 48.2 20.8 8.0X | ||
| Array: Float with unsafe:false 6 6 0 178.9 5.6 29.6X |
There was a problem hiding this comment.
Double in throughput 🤔
Are we are expecting this kind of improvement on a hotfix bump I wonder.
There was a problem hiding this comment.
@hiufung-kwok Judging from the historical update records, the current data seems relatively normal, whereas the previous update results may have had some unknown errors. It is also possible that there are certain unstable factors affecting the results of this test case, which could be worth investigating.
|
friendly ping @yaooqinn |
|
Merged to master. Can you backport @LuciferYang since it's a bugfix release of Kryo? |
What changes were proposed in this pull request?
This pr aims to upgrade
kryo-shadedfrom 4.0.2 to 4.0.3.Why are the changes needed?
This version brings some bug fixes and performance improvements related to chunked encoding:
full changes as follows:
Does this PR introduce any user-facing change?
No
How was this patch tested?
Was this patch authored or co-authored using generative AI tooling?
No