Skip to content

Conversation

@dayshah
Copy link
Contributor

@dayshah dayshah commented Aug 25, 2025

Why are these changes needed?

Today our gRPC servers have polling thread(s). They end up not actually doing much work because the actual copying of the requests into the proto objects is offloaded to the main service io_context thread(s). When sending relatively large amounts of inlined object data (~1-5mb) we found that there can be a bottleneck on actually receiving the args because the args are copied into the proto on the single-threaded io_context that is also doing a lot of other work. The # of polling threads meanwhile can be scaled up without concurrency issues and they're much less busy.

This could also help offload work off the GCS's main io context + fully utilize the # of object manager threads.

@dayshah dayshah added the go add ONLY when ready to merge, run all tests label Aug 25, 2025
@dayshah dayshah changed the title [core] Move create call / receiving off of execution io_context [core] Move create call / receiving request off of execution io_context Aug 25, 2025
@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

@github-actions github-actions bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Sep 10, 2025
@dayshah dayshah added unstale A PR that has been marked unstale. It will not get marked stale again if this label is on it. and removed stale The issue is stale. It will be closed within 7 days unless there are further conversation labels Sep 15, 2025
Signed-off-by: dayshah <[email protected]>
Signed-off-by: dayshah <[email protected]>
Signed-off-by: dayshah <[email protected]>
@dayshah dayshah force-pushed the move-server-create-call branch from 19af7c8 to 3910112 Compare November 23, 2025 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

go add ONLY when ready to merge, run all tests unstale A PR that has been marked unstale. It will not get marked stale again if this label is on it.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant