Define the API for LoadBalancer to add headers

[zhangkun83]

GRPCLB needs this to add the per-client LB token to every RPC.

Metadata injection is usually done in interceptors, but LoadBalancer cannot use interceptor, because it returns transports, not channels. Since LoadBalancer doesn't need the full power of interceptor anyway, we may just figure out an API that allows LoadBalancer to change headers.

[zhangkun83]

Because Metadata is mutable, we can pass the headers to pickTransport() in addition to the affinity, so that LoadBalancer can mutate it:

LoadBalancer {
  T pickTransport(Attributes affinity, Metadata headers);
}

If LoadBalancer has returned an interim transport (delayed transport), and sets real transports to it later, the headers mutation needs to happen when setting the real transport. This can be probably comined with the changes proposed for #2302, and will get us this:

TransportPicker<T> {
  T pickTransport(Attributes affinity, Metadata headers);
}

[jhspaybar]

Will this modified state be available to interceptors? The specific usage I'm thinking about is allowing a tracing interceptor to add metadata to the trace about the server target picked.

[ejona86]

@jhspaybar, I don't think interceptors will be able to see that. All the interceptors have run by the time the LoadBalancer makes its decision.

[Xorlev]

@ejona86 I was building a interceptor for performing backup requests. While I can do successfully send and manage backup requests, there's no way for me to signal to the load balancer that I'd like it to choose different machine from the address group if available. Or, more advanced, send the original request's target address to the backup machine for it to handle request cancellation.

Do you think there's a clean way to expose call/balancer state in gRPC? Or would features like hedging have to be built into the framework to facilitate state sharing?

[ejona86]

@Xorlev, we're already starting work (not in Java yet, but in C) on hedging and tied requests as part of our retry API. That's probably the best place to support supplementing the RPC, at least for your use case.

[Xorlev]

@ejona86 Excellent news, thanks for the response. :)

[ejona86]

@Xorlev, oh, and I should mention, that we're targeting Q1 for the work.

[zhangkun83]

The LBv2 API has resolved this, though won't make it into 1.1.