DWARF2 Queue Guard Internal Error on run

[christiankissig]

What are the steps to reproduce this issue?

I'm using

> nvim --version                                                                                                                                                                                                                
NVIM v0.10.0-dev-28f54a7
Build type: RelWithDebInfo
LuaJIT 2.1.1694751058
 ~/.local/share/nvim/site/pack/packer/start/nvim-gdb > git rev-parse HEAD
6fd4eb53ac25fda0c2f1a60457c5f0592a0e95ad
> gdb --version
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1

For context, I'm working on https://github.com/llvm/llvm-project

1. Open Neovim
2. Execute :GdbStart gdb --args /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests --gtest_filter=AArch64SelectionDAGTest.ComputeKnownBits_SUBC in Neovim
3. Set break point (gdb) break AArch64SelectionDAGTest.cpp:278 in nvim-gdb buffer
4. Run (gdb) run in nvim-gdb buffer

What happens?

DWARF2 queue guard errors out with an internal error:

    1 Reading symbols from /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/./CodeGenTests...
    2 (gdb) break AArch64SelectionDAGTest.cpp:278
    3 Breakpoint 1 at 0x5fe405: file /home/ubuntu/workspace/llvm-project/llvm/unittests/CodeGen/AArch64SelectionDAGTest.cpp, line 278.
    4 (gdb) run
    5 Starting program: /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests --gtest_filter=AArch64SelectionDAGTest.ComputeKnownBits_SUBC
    6 [Thread debugging using libthread_db enabled]
    7 Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
    8 Note: Google Test filter = AArch64SelectionDAGTest.ComputeKnownBits_SUBC
    9 [==========] Running 1 test from 1 test suite.
   10 [----------] Global test environment set-up.
   11 [----------] 1 test from AArch64SelectionDAGTest
   12 [ RUN      ] AArch64SelectionDAGTest.ComputeKnownBits_SUBC
   13
   14 Breakpoint 1, llvm::AArch64SelectionDAGTest_ComputeKnownBits_SUBC_Test::TestBody (/build/gdb-ZgDh0V/gdb-12.1/gdb/dwarf2/read.c:1498: internal-error: dwarf2_queue_guard: Assertion `!m_per_objfile->per_bfd->queue.has_value ()' failed.
   15 A problem internal to GDB has been detected,
   16 further debugging may prove unreliable.
   17 ----- Backtrace -----
   18 0x55c93435f077 ???
   19 0x55c9346c3a64 ???
...
   56 0x7f126d0857bc ???
   57 0x7f126d01758a ???
   58 0x7f126c094b42 start_thread
   59         ./nptl/pthread_create.c:442
   60 0x7f126c1269ff ???
   61         ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
   62 0xffffffffffffffff ???
   63 ---------------------
   64
   65 This is a bug, please report it.  For instructions, see:
   66 <https://www.gnu.org/software/gdb/bugs/>.
   67
   68
   69 [Process exited 134]

What were you expecting to happen?

In the terminal the same succeeds

> gdb --args /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests --gtest_filter=AArch64SelectionDAGTest.ComputeKnownBits_SUBC
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests...
(gdb) break AArch64SelectionDAGTest.cpp:278
Breakpoint 1 at 0x5fe405: file /home/ubuntu/workspace/llvm-project/llvm/unittests/CodeGen/AArch64SelectionDAGTest.cpp, line 278.
(gdb) run
Starting program: /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests --gtest_filter=AArch64SelectionDAGTest.ComputeKnownBits_SUBC
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Note: Google Test filter = AArch64SelectionDAGTest.ComputeKnownBits_SUBC
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from AArch64SelectionDAGTest
[ RUN      ] AArch64SelectionDAGTest.ComputeKnownBits_SUBC

Breakpoint 1, llvm::AArch64SelectionDAGTest_ComputeKnownBits_SUBC_Test::TestBody (this=0x5555621bcfa0) at /home/ubuntu/workspace/llvm-project/llvm/unittests/CodeGen/AArch64SelectionDAGTest.cpp:278
278       SDLoc Loc;
(gdb)

Any logs, error output, etc?

To obtain the logs (nvimgdb.log, proxy.log, spy_ui.log), launch neovim as follows:
cd <plugin_dir>/test; CI=1 ./nvim

Command doesn't exist. CI=1 ./nivm.py fails with https://gist.github.com/christiankissig/e8125e2f4a14de2150205794411c6d5f

If it’s long, please paste to https://gist.github.com/ and insert the link here.

See log output above

Any other comments?

No

[sakhnik]

Thanks for reporting. It'll take some time and effort to investigate.

[sakhnik]

@christiankissig, I was able to reproduce the issue with your instructions. And gdb doesn't crash anymore for me after the fix, although in x64. Could you please verify the bug has been fixed in your environment?

[christiankissig]

Thank you for the quick fix @sakhnik . The issue is fixed, but I'm seeing another issue

    1 Reading symbols from /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/./CodeGenTests...
    2 (gdb) break KnownBits.cpp:90
    3 Breakpoint 1 at 0x442c2ae: file /home/ubuntu/workspace/llvm-project/llvm/lib/Support/KnownBits.cpp, line 90.
    4 (gdb) run
    5 Starting program: /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests --gtest_filter=AArc
    6 h64SelectionDAGTest.ComputeKnownBits_USUBO_CARRY
    7 /build/gdb-ZgDh0V/gdb-12.1/gdb/frame.c:3052: internal-error: frame_cleanup_after_sniffer: Assertion `frame->pr
    8 ologue_cache == NULL' failed.
    9 A problem internal to GDB has been detected,
   10 further debugging may prove unreliable.
   11 ----- Backtrace -----
   12 0x55bcb330c077 ???
   13 0x55bcb3670a64 ???
...
   38 0x7f607801758a ???
   39 0x7f6077294b42 start_thread
   40         ./nptl/pthread_create.c:442
   41 0x7f60773269ff ???
   42         ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
   43 0xffffffffffffffff ???
   44 ---------------------
   45
   46 This is a bug, please report it.  For instructions, see:
   47 <https://www.gnu.org/software/gdb/bugs/>.

I seem to have above commit

 ubuntu@ip-172-31-16-55  ~/.local/share/nvim/site/pack/packer/start/nvim-gdb   master  git log --pretty=oneline 
69d055aad50adb80a5f0b1ddeeef05dce92871c2 (HEAD -> master, origin/master, origin/HEAD) Merge branch 'devel'
302f075756b7bbdb6cd291f72a29375cf79a5d04 (origin/devel) Merge branch 'lua' into devel
12c04360779a6b552e8616523a3425e0a3706514 (origin/lua) Try disabling failing test: fails only in GH actions
78bf4dfa68b5995e8037dca7e3371aaaa1ad6747 Fix #201: track the state of the inferior with events
5b5dc35f58324256dc62285c77c43353866606e7 Update spy_ui to the latest protocol
428552fd5e47623c268bc150357be126bf82295c (grafted) Fix typo
6fd4eb53ac25fda0c2f1a60457c5f0592a0e95ad (grafted) Merge branch 'devel'
993e43e8e3dced7915bf9be97065da5464daef63 (grafted) Fix #198 use socket.exception appropriately

From console this GDB works fine

 ubuntu@ip-172-31-16-55  ~/workspace/llvm-project   main ● ?  gdb -q --args /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests --gtest_filter=AArch64SelectionDAGTest.ComputeKnownBits_UADDO_CARRY
Reading symbols from /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests...
(gdb) break KnownBits.cpp:90
Breakpoint 1 at 0x442c2ae: file /home/ubuntu/workspace/llvm-project/llvm/lib/Support/KnownBits.cpp, line 90.
(gdb) run
Starting program: /home/ubuntu/workspace/llvm-project/build/unittests/CodeGen/CodeGenTests --gtest_filter=AArch64SelectionDAGTest.ComputeKnownBits_UADDO_CARRY
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Note: Google Test filter = AArch64SelectionDAGTest.ComputeKnownBits_UADDO_CARRY
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from AArch64SelectionDAGTest
[ RUN      ] AArch64SelectionDAGTest.ComputeKnownBits_UADDO_CARRY
[       OK ] AArch64SelectionDAGTest.ComputeKnownBits_UADDO_CARRY (40 ms)
[----------] 1 test from AArch64SelectionDAGTest (40 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (41 ms total)
[  PASSED  ] 1 test.
[Inferior 1 (process 28317) exited normally]
(gdb)

Do you want me to open another issue?

[christiankissig]

Btw this not running on aarch64, just a cross-compile scenario being tested:

> uname -m
x86_64

[sakhnik]

I'm reopening the same ticket. It looks like the issue is more severe than I anticipated. A python script is run inside gdb to fetch information about the current line and breakpoints: https://github.com/sakhnik/nvim-gdb/blob/master/lib/gdb_commands.py . And because a separate thread is forked to handle external commands, I can easily imagine how different kinds of data races can occur.
So let me take a closer look at gdb.

I'm not able to reproduce the issue this time. How do you build llvm?

[christiankissig]

I'm sorry @sakhnik , I've missed this message in my inbox.

Thanks for the deep dive! I'm building LLVM as follows

❯ cmake -S llvm -B build -G Ninja -DCMAKE_BUILD_TYPE=Debug
❯ ninja -C build