compatibility with uvloop

[wyn1995]

Reproduction Steps:

pip3 install uvloop pyroute2

import uvloop
import pyroute2

uvloop.install()
pyroute2.IPRoute()

Error Traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/venv/lib/python3.11/site-packages/pyroute2/iproute/linux.py", line 2653, in __init__
    self.asyncore.event_loop.run_until_complete(
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/root/venv/lib/python3.11/site-packages/pyroute2/netlink/nlsocket.py", line 281, in setup_endpoint
    await self.event_loop.create_datagram_endpoint(
  File "uvloop/loop.pyx", line 3056, in create_datagram_endpoint
  File "uvloop/handles/udp.pyx", line 97, in uvloop.loop.UDPTransport.open
ValueError: cannot open a UDP handle, invalid family 16

[MorningLightMountain713]

I am also getting this error - was wondering why my dev code was working - dev code is using asyncio.

Edit - Ubuntu 24.10

Package	Version
python	3.12.7
pyroute2	0.9.2
uvloop	0.21.0

│   680 │   │   async with AsyncIPRoute() as ipr:                                                                                                       │
│ ❱ 681 │   │   │   async for link in await ipr.link("dump"):                                                                                           │
│   682 │   │   │   │   links[link.get("index")] = {                                                                                                    │
│   683 │   │   │   │   │   "state": link.get("state"),                                                                                                 │
│   684 │   │   │   │   │   "address": link.get("address"),                                                                                             │
│                                                                                                                                                       │
│ ╭───────────────────────────────────── locals ─────────────────────────────────────╮                                                                  │
│ │ filtered_routes = []                                                             │                                                                  │
│ │             ipr = <pyroute2.iproute.linux.AsyncIPRoute object at 0x71eb483f0ad0> │                                                                  │
│ │           links = {}                                                             │                                                                  │
│ │            self = NetworkScreen()                                                │                                                                  │
│ ╰──────────────────────────────────────────────────────────────────────────────────╯                                                                  │
│                                                                                                                                                       │
│ /home/fluxadm/dev/flux_config/.venv/lib/python3.12/site-packages/pyroute2/iproute/linux.py:1845 in link                                               │
│                                                                                                                                                       │
│   1842 │   │   request = NetlinkRequest(                                                                                                              │
│   1843 │   │   │   self, ifinfmsg(), command, command_map, dump_filter, arguments                                                                     │
│   1844 │   │   )                                                                                                                                      │
│ ❱ 1845 │   │   await request.send()                                                                                                                   │
│   1846 │   │   if command == 'dump':                                                                                                                  │
│   1847 │   │   │   return request.response()                                                                                                          │
│   1848 │   │   return [x async for x in request.response()]                                                                                           │
│                                                                                                                                                       │
│ ╭───────────────────────────────────── locals ──────────────────────────────────────╮                                                                 │
│ │   arguments = {}                                                                  │                                                                 │
│ │     command = 'dump'                                                              │                                                                 │
│ │ command_map = {                                                                   │                                                                 │
│ │               │   'set': (16, 'req'),                                             │                                                                 │
│ │               │   'update': (19, 'create'),                                       │                                                                 │
│ │               │   'add': (16, 'create'),                                          │                                                                 │
│ │               │   'del': (17, 'req'),                                             │                                                                 │
│ │               │   'property_add': (108, 'append'),                                │                                                                 │
│ │               │   'property_del': (109, 'req'),                                   │                                                                 │
│ │               │   'remove': (17, 'req'),                                          │                                                                 │
│ │               │   'delete': (17, 'req'),                                          │                                                                 │
│ │               │   'dump': (18, 'dump'),                                           │                                                                 │
│ │               │   'get': (18, 'get')                                              │                                                                 │
│ │               }                                                                   │                                                                 │
│ │ dump_filter = {}                                                                  │                                                                 │
│ │       kwarg = {}                                                                  │                                                                 │
│ │     request = <pyroute2.netlink.nlsocket.NetlinkRequest object at 0x71eb484020f0> │                                                                 │
│ │        self = <pyroute2.iproute.linux.AsyncIPRoute object at 0x71eb483f0ad0>      │                                                                 │
│ ╰───────────────────────────────────────────────────────────────────────────────────╯                                                                 │
│                                                                                                                                                       │
│ /home/fluxadm/dev/flux_config/.venv/lib/python3.12/site-packages/pyroute2/netlink/nlsocket.py:619 in send                                             │
│                                                                                                                                                       │
│   616 │   │   return True                                                                                                                             │
│   617 │                                                                                                                                               │
│   618 │   async def send(self):                                                                                                                       │
│ ❱ 619 │   │   await self.prepare()                                                                                                                    │
│   620 │   │   if await self.proxy():                                                                                                                  │
│   621 │   │   │   return len(self.msg.data)                                                                                                           │
│   622 │   │   count = 0                                                                                                                               │
│                                                                                                                                                       │
│ ╭────────────────────────────────── locals ──────────────────────────────────╮                                                                        │
│ │ self = <pyroute2.netlink.nlsocket.NetlinkRequest object at 0x71eb484020f0> │                                                                        │
│ ╰────────────────────────────────────────────────────────────────────────────╯                                                                        │
│                                                                                                                                                       │
│ /home/fluxadm/dev/flux_config/.venv/lib/python3.12/site-packages/pyroute2/netlink/nlsocket.py:593 in prepare                                          │
│                                                                                                                                                       │
│   590 │   │   │   return all(matches)                                                                                                                 │
│   591 │                                                                                                                                               │
│   592 │   async def prepare(self):                                                                                                                    │
│ ❱ 593 │   │   await self.sock.setup_endpoint()                                                                                                        │
│   594 │   │   self.msg.encode()                                                                                                                       │
│   595 │   │   self.sock.msg_queue.ensure_tag(self.msg_seq)                                                                                            │
│   596 │   │   if self.parser is not None:                                                                                                             │
│                                                                                                                                                       │
│ ╭────────────────────────────────── locals ──────────────────────────────────╮                                                                        │
│ │ self = <pyroute2.netlink.nlsocket.NetlinkRequest object at 0x71eb484020f0> │                                                                        │
│ ╰────────────────────────────────────────────────────────────────────────────╯                                                                        │
│                                                                                                                                                       │
│ /home/fluxadm/dev/flux_config/.venv/lib/python3.12/site-packages/pyroute2/netlink/nlsocket.py:281 in setup_endpoint                                   │
│                                                                                                                                                       │
│   278 │   │   if getattr(self.local, 'transport', None) is not None:                                                                                  │
│   279 │   │   │   return                                                                                                                              │
│   280 │   │   self.local.transport, self.local.protocol = (                                                                                           │
│ ❱ 281 │   │   │   await self.event_loop.create_datagram_endpoint(                                                                                     │
│   282 │   │   │   │   lambda: CoreDatagramProtocol(                                                                                                   │
│   283 │   │   │   │   │   self.connection_lost,                                                                                                       │
│   284 │   │   │   │   │   self.enqueue,                                                                                                               │
│                                                                                                                                                       │
│ ╭─────────────────────────────── locals ────────────────────────────────╮                                                                             │
│ │ self = <pyroute2.iproute.linux.AsyncIPRoute object at 0x71eb483f0ad0> │                                                                             │
│ ╰───────────────────────────────────────────────────────────────────────╯                                                                             │
│                                                                                                                                                       │
│ in create_datagram_endpoint:3056                                                                                                                      │
│                                                                                                                                                       │
│ in uvloop.loop.UDPTransport.open:97                                                                                                                   │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: cannot open a UDP handle, invalid family 16

[svinota]

Looked at the module, and it looks like uvloop doesn't support datagram endpoints over netlink sockets, while asyncio does.

I'm not sure if I can fix this on my side.

If you have any ideas, let me know.

[kucejan]

It should be possible to work around it by passing an existing asyncio-based event loop using the use_event_loop argument.

import uvloop
import pyroute2
import selectors

uvloop.install()

selector = selectors.SelectSelector()
loop = asyncio.SelectorEventLoop(selector)

pyroute2.IPRoute(use_event_loop=loop)

However, IPRoute() does not respect the argument and creates the loop on its own. It is probably broken since this 389124f#diff-5c2c3701c4d747e66e10fb1b1f293347bfeba008020d227d269f5907a7a4990aL199 refactoring step.

The property is not assigned (self.local.event_loop = use_event_loop) anywhere.

[svinota]

There is some issue with using in IPRoute an externally set up event loop, but it needs to be fixed a long time ago already.

[svinota]

@kucejan I can confirm that your way is working using the experimental branch #1398

I will merge the branch in the master by Saturday after adding some more tests and documentation.

[kucejan]

@svinota That's great news. Thanks for your efforts.