Required networking functionality in parachain subsystems

[haikoschol]

From what I've seen so far working on the parachain subsystems, the main networking functionality required there is:

1. Sending requests to specific peers on specific protocols
2. Receiving requests from any peer on specific protocols
3. Processing overseer events like PeerConnected
4. Authority Discovery

1. Sending requests to specific peers on specific protocols

We already have an implementation for this, modeled after that in polkadot-sdk:

// SendRequests is a subsystem message for sending requests over a request response protocol.
type SendRequests struct {
	Requests       []*OutgoingRequest
	IfDisconnected IfDisconnectedBehavior
}

// OutgoingRequest contains all data required to send a request over a request response protocol and receive the result.
type OutgoingRequest struct {
	Recipient parachaintypes.RecipientID
	Payload   ReqProtocolMessage
	Result    chan ReqRespResult

	ctx    context.Context
	cancel context.CancelFunc
}

However, this implementation has some limitations:

• Parameters like request timeout and maximum response size are hard-coded.
• Since there is no way to check whether a peer is connected, the IfDisconnectedBehavior attribute is ignored.
• Although the idea behind using parachaintypes.RecipientID instead of a PeerID is to allow sending requests to a parachaintypes.AuthorityDiscoveryID, this is not implemented yet.

So far, this is used in availability distribution, but it will also be used in collator protocol, statement distribution and availability recovery, for example.

2. Receiving requests from any peer on specific protocols

This functionality is not implemented yet. So far we only have an interface as a dependency in subsystems that need it, which allows registering a request handler:

type Network interface {
	RegisterRequestHandler(subprotocolID protocol.ID, handler network.RequestHandler)
}

3. Processing overseer events like PeerConnected

These events are not yet dispatched by the overseer. Currently, the overseer knows two types of messages, broadcast and 1-to-1. Broadcast messages are "block events" like ActiveLeavesUpdateSignal and BlockFinalizedSignal. Although there is a Subsystem interface which includes methods for handling these, the overseer does not actually call them. Instead it sends these messages over the channel it has for each subsystem in OverseerSystem.broadcast().

For 1-to-1 messages, the same channel is used. The logic which subsystem should receive which message is hard-coded in a switch statement in OverseerSystem.processMessage().

An example for a subsystem that needs to process networking events like PeerConnected, NewGossipTopology or PeerViewChangeEvent is bitfield distribution.

We need to come up with a way for the overseer to receive and dispatch these events. Alternatively we could also give each subsystem a dependency on which it can subscribe to the events it needs, but that would require changing the implementation of the affected subsystems, since they won't get all messages via the overseer channel anymore. However, this is already true for subsystems that need to receive requests. In any case, these events need to somehow make their way from the networking layer to the subsystems.

4. Authority Discovery

There are at least two ways in which authority discovery is used inside parachain subsystems. One is already described under point 1 above. In some cases a subsystem wants to send a request to a specific peer but only has the authority ID. In that case we can handle the lookup in the code that processes OutgoingRequest. This should be easy at least in the case where the peer ID for a given authority ID is in the cache. If a network round trip to find it is required, it might take a bit more coordination.

The other way authority discovery is used is by calling the methods on the authority discovery service (e.g. GetAddressesByAuthorityID()). It looks like this is only done in the gossip support subsystem, which then sends a UpdatedAuthorityIDs message to other subsystems via the overseer (This is true for polkadot-sdk. In Gossamer those events are not dispatched yet, see point 3).

[EclesioMeloJunior]

In any case, these events need to somehow make their way from the networking layer to the subsystems.

Can we modify the network bridge subsystem to register all these protocols and then make it responsible to dispatch the messages through overseer?

Basically we leave the responsability on top of a specific subsystem, can be a brand new subsystem just to receiving the network protocol messages and this subsystem will know how to decode and to which inner subsystem forward the message