After running the command in the cloud environment, the error keeps looping. #262
Description
Subject
Issue type
- Bug report
- Feature request
Configuration:
- Product version
XLIO INFO : XLIO_VERSION: 3.40.2-1 Release built on Oct 7 2024 17:14:06 -*- DEBUG -*-
- OS
~# uname -a
Linux 10-20-1-50 6.5.0-18-generic #18~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 7 11:40:03 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
~# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
- OFED
~# ofed_info -s
MLNX_OFED_LINUX-24.10-0.7.0.0:
- Hardware
ConnectX-7
~# mlxconfig -d 0d:00.0 q
Device #1:
----------
Device type: ConnectX7
Name: MCX75310AAS-NEA_Ax
Description: NVIDIA ConnectX-7 HHHL Adapter card; 400GbE / NDR IB (default mode); Single-port OSFP; PCIe 5.0 x16; Crypto Disabled; Secure Boot Enabled;
Device: 0d:00.0
Actual behavior:
~# LD_PRELOAD=libxlio-debug.so sockperf sr -i 172.17.13.50
XLIO INFO : ---------------------------------------------------------------------------
XLIO INFO : XLIO_VERSION: 3.40.2-1 Release built on Oct 7 2024 17:14:06 -*- DEBUG -*-
XLIO INFO : Cmd Line: sockperf sr -i 172.17.13.50
XLIO INFO : OFED Version: MLNX_OFED_LINUX-24.10-0.7.0.0:
XLIO INFO : ---------------------------------------------------------------------------
XLIO INFO : Log Level INFO [XLIO_TRACELEVEL]
XLIO INFO : ---------------------------------------------------------------------------
XLIO WARNING: ************************************************************************
XLIO WARNING: Your current max locked memory is: 16878186496. Please change it to unlimited.
XLIO WARNING: Set this user's default to `ulimit -l unlimited`.
XLIO WARNING: Read more about this topic in the XLIO's User Manual.
XLIO WARNING: ************************************************************************
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)It looks like it's looping forever and my sockperf tests are not succeeding.
~# xlio_stats
xliostat: stats for application: sockperf with pid: 1717047
fd ------------ total offloaded ------------- --------- total os ----------
pkt Kbyte eagain error poll% pkt Kbyte eagain error
Eventually the test will fail and XLIO will crash.
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali07d2d70bb52/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
XLIO ERROR : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali07d2d70bb52/address (errno 2 No such file or directory)
XLIO ERROR : ******************************************************************************************************
XLIO ERROR : * Failed registering a memory region of size 2147483648 bytes
XLIO ERROR : * (errno=5 Input/output error)
XLIO ERROR : * Could be due to lack of locked memory in kernel.
XLIO ERROR : * Please check max allowed locked memory (ulimit -l)
XLIO ERROR : ******************************************************************************************************
XLIO WARNING: allocator[0x556329eb24a8]:270:register_memory_single() Failure during memory registration on dev mlx5_10 addr=0x7fd637fff000 size=2147483648
XLIO WARNING: allocator[0x556329eb24a8]:272:register_memory_single() This might happen due to low MTT entries. Please refer to README for more info
XLIO ERROR : socket_internal XLIO failed to start errno: Input/output error
sockperf: ERROR: socket(AF_INET4/6/AF_UNIX, SOCK_x) (errno=5 Input/output error)
sockperf: ERROR: program exits because of an error. (errno=5 Input/output error)Expected behavior:
The sockperf test can see the acceleration effect.
Steps to reproduce:
server:
LD_PRELOAD=libxlio.so sockperf sr -i 172.17.13.50
client:
LD_PRELOAD=libxlio.so sockperf pp -i 172.17.13.50