Skip to content

Conversation

alexeykudinkin
Copy link
Contributor

Why are these changes needed?

  1. Addressing serializability issues reported in [data -- read_iceberg] pickling error on UDF for dataset.groupby.map_batches #54280
  2. Avoid back-and-forth conversion from block to batch and back
  3. Expose zero_copy_batch option

Related issue number

Addresses #54280

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@alexeykudinkin alexeykudinkin requested a review from a team as a code owner July 9, 2025 03:16
@alexeykudinkin alexeykudinkin added the go add ONLY when ready to merge, run all tests label Jul 9, 2025
Signed-off-by: Alexey Kudinkin <[email protected]>
Signed-off-by: Alexey Kudinkin <[email protected]>
Copy link
Contributor

@raulchen raulchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alexeykudinkin alexeykudinkin merged commit b5631ea into ray-project:master Jul 9, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
go add ONLY when ready to merge, run all tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[data -- read_iceberg] pickling error on UDF for dataset.groupby.map_batches
2 participants