Impossible to access `PutPayload` variant of `HTTPRequestBody` from custom `HttpConnector`

[kylebarron]

Describe the bug

When trying to implement a custom HttpConnector, one needs to implement HttpService. That service needs to operate on HttpRequest, whose type of body is HttpRequestBody.

The implementation of HttpRequestBody is an enum with either a single or multiple buffers:

arrow-rs-object-store/src/client/http/body.rs

Lines 111 to 112 in 12ef9bc

	Bytes(Bytes),
	PutPayload(usize, PutPayload),

But the public API of HttpRequestBody presents no way to access the PutPayload variant. There's an as_bytes public method, but that returns None for Inner::PutPayload.

To Reproduce

(maybe this is a feature request instead of a bug report?)

Expected behavior

Should be able to access all internal variants through the public API.

Additional context

I'm prototyping integrating the HttpConnector into obstore — Python bindings to object_store — so that someone can define an arbitrary Python-based HTTP client to make the requests initiated by object_store.

[tustvold]

The intention is for implementations to use https://docs.rs/http-body/1.0.1/http_body/trait.Body.html

[kylebarron]

The HTTPRequestBody is I think always known upfront and is never a stream, so it would be nice if there were a synchronous way to access the payload instead of having to go through Body

[tustvold]

I think we need to keep Inner crate-private, in particular the index tracking on the PutPayload variant would be a little odd to expose.

I can see a few options:

• Add a as_payload(&self) -> Option<&PutPayload>
• Add a From<HttpRequestBody> for PutPayload
• Add a impl Iterator<Item=&Bytes> for &HttpRequestBody

However, these would all potentially constrain us down the line. Whilst we don't currently support streaming writes, this isn't necessarily always going to be the case. #504 #281

I'm not familiar with where pyO3 's async story has gotten, but is this a hard blocker or just a nice-to-have?