Document IPV6 Connectivity

[AgeManning]

With the introduction of #4046, Lighthouse mostly supports ipv6 connectivity. However the end-user API has some caveats that we should document.

This issue is to add documentation to the book about these caveats.

[totoCZ]

Why is "--listen-address '0.0.0.0' --listen-address '::'" not the default?

[totoCZ]

"udp4"=> ?local_enr.udp4(), "tcp4" => ?local_enr.tcp6()

https://github.com/sigp/lighthouse/blob/stable/beacon_node/lighthouse_network/src/discovery/mod.rs#LL210C41-L210C97

Designed to be impossible to configure and then it still tries to confuse you.
I would scrap this implementation and use a single socket, enabled by default.

When you set --enr-address manually (it's not clear if the ext. IP is resolved automatically or not, so I set it), it asks for --enr-udp-port.
Except the doc says that one takes the default value of --port. ("The --port flag is used if this is not set.")
But my --port is not set either and it's clearly not working. (oh, wait, that's only for TCP! UDP does not take defaults.)

So my lightweight config to be sure ended up as

  --listen-address '0.0.0.0' \
  --listen-address '::' \
  --port 9000 \
  --port6 9090 \
  --enr-address 185.175.90.195 \
  --enr-address 2a10:4740:45:1:a0d6:8aff:fed2:7ec8 \
  --enr-udp-port 9000 \
  --enr-udp6-port 9090 \
  --enr-tcp-port 9000 \
  --enr-tcp6-port 9090

[AgeManning]

Why is "--listen-address '0.0.0.0' --listen-address '::'" not the default?

Ipv6 socket is not enabled by default because most people have not got ipv6 configured by default. We prefer not to open sockets that users may be unaware of by default. Furthermore, ipv6 is mostly experimental on the Ethereum network, because not many other client teams/boot nodes support ipv6 so you'll have difficulty finding and maintaining connections on ipv6. So mainly because its an experimental feature that is not fully operational, we do not have it enabled by default.

The line you pointed out is a bug. I'll make a PR to fix it.

We have used a single socket with mapped addresses in discovery. Libp2p enabled multiple sockets. We've opted to use multiple sockets to make things easier to debug/reason about but also to give the greatest adoption. We can also add mapped ipv6 sockets on after (seeing as this is the implementation we originally went with in discovery).

The IP addresses should get resolved automatically if you don't set the --enr-address field. This is the default. The catch is you need to find enough nodes that agree on what your external address is. As there probably are no dual-stack or ipv6-only nodes on the network, you probably wont resolve your ipv6 address.

As for the --port flag. You need to set which port to listen on. You can listen on port 9000 for both udp/tcp and udp6 and tcp6 by leaving it blank or specifying --port 9000. If you want to specify these ports manually to arbitrary values, you can do so via the flags individually as you've done in your sample config.

Now, listening on a port, is different to advertising which port users can connect to you. For example, you can listen on port 9000, but your router may port-forward 10000 to that local port. So it's not wise to advertise the local port you are listening on by default, because in general this is not the port any one can connect to you. We advertise the ports that people can connect to us via our ENR. You would have seen that your ENR ports would be set to None by default even though you may be listening on port 9000.
When you advertise None in IP and port fields, this signifies to others that you are still trying to find your external ports. Once discovery has done its job, these fields will get updated. To get a better understanding of this, you may like to read: https://github.com/sigp/discv5#addresses-in-enrs

This issue is about updating our lighthouse book to explain these nuisances. I'll tag @divagant-martian in case they would like to add anything I may have missed here.

[divagant-martian]

Thanks for weighting in,

First and foremost

Designed to be impossible to configure and then it still tries to confuse you. I would scrap this implementation and use a single socket, enabled by default.

Please use an appropriate tone within this repository to make suggestions or bug reports.

libp2p has opted to disable mapped addresses for good reasons, and so have we. We won't be adding a single port implementation.

The cli flags for dual stack are tricky since we need to ensure backwards compatibility. It's not ideal, but it's the best effort solution to ensure this is achievable without breaking user's configurations and without enabling by default unexpected behaviours.

As @AgeManning has mentioned, this feature is very much experimental. That said, the behaviour of --port and --enr-address has been the same for a while and it's documented. Please refer to https://lighthouse-book.sigmaprime.io/advanced_networking.html

What your configuration ends up being is exactly what we expect. This will be documented when this issue is addressed

[totoCZ]

Ipv6 socket is not enabled by default because most people have not got ipv6 configured by default.

ehm, it’s not 1993 anymore.

Please use an appropriate tone within this repository to make suggestions or bug reports.

huehuehue

Or am I getting this wrong? Did you you actually mean lighthouse bn —please-use-appropriate-tone? In that case sorry, I missed that before.

I will add it to my config.

The cli flags for dual stack are tricky since we need to ensure backwards compatibility. It's not ideal, but it's the best effort solution to ensure this is achievable without breaking user's configurations and without enabling by default unexpected behaviours.

i think you can put that into readme.

anyway it’s clear someone just could do it better. geth is a great example of a client that has a normal config, even if it doesn’t discover any peers yet.
imagine if nginx had 56 flags to set up a socket ahaha and you spend an hour doing it.
take care guys.

[michaelsproul]

Or am I getting this wrong? Did you you actually mean lighthouse bn —please-use-appropriate-tone? In that case sorry, I missed that before.

No buddy, if you continue to be rude we will stop talking to you. You have no entitlement to our time, or the time of any open source maintainer. We are doing our best to build secure, free software and don't need to be critiqued by you. If you think you can do better, open a pull request or build your own client.

[AgeManning]

anyway it’s clear someone just could do it better. geth is a great example of a client that has a normal config, even if it doesn’t discover any peers yet.
imagine if nginx had 56 flags to set up a socket ahaha and you spend an hour doing it.

Your lack of understanding of how the config works, does not make it a bad config.

As I was explaining before, its pretty easy to set up an ipv6 dual stack node. See here:

lighthouse bn --listen-address 0.0.0.0 --listen-address '::' 

I'm sorry it took you an hour to add these two flags but for others reading this issue in the future, it's pretty straight forward to add an ipv6 socket (and also a far-cry from the claimed 56 flags).

If you think you can improve the config or these two flags are too much for you, as @michaelsproul said, feel free to open a PR.