creating temporary files takes a long time on Java 11 #1165
Description
As I was running java unit tests on Java 11 I noticed that one of the test cases that are part of modules/java-tests/tests/src/main/java/io/osv/LoggingWithoutIsolationTest.java takes a long time (around 10 seconds) when run on Java 11. The same test runs very quickly on Java 8.
After adding printouts I have isolated this Java code taking a long time:
File file = File.createTempFile("test", null);
See https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/File.html#createTempFile(java.lang.String,java.lang.String) for what this does.
Eventually the program finishes and one can see this (I added the line above to the Java hello worlds example):
./scripts/run.py -e '/usr/bin/java -cp /java-example Hello !' -c 1
OSv v0.55.0-298-g18555601
eth0: 192.168.122.15
Booted up in 174.74 ms
Cmdline: /usr/bin/java -cp /java-example Hello !
Hello, World! //->> This is the last line after which it waits for 10 seconds or so
fpathconf() stubbed
-> After File.createTempFile()
After connecting with gdb I see this:
(gdb) bt
#0 fill_ts<long, std::ratio<1, 1000000000> > (ts=0x2000197019a0, d=...) at /usr/include/c++/10/chrono:422
#1 clock_gettime (clk_id=<optimized out>, ts=0x2000197019a0) at libc/time.cc:81
#2 0x0000100000c7e580 in os::javaTimeNanos() ()
#3 0x0000200010cd0ce8 in ?? ()
#4 0x0000000800356780 in ?? ()
#5 0x0000000000000000 in ?? ()
(gdb) osv info threads
id address name cpu status vruntime total_cpu_time location
1 (0xffff800000016040) reclaimer cpu0 waiting 2.97268e-25 562 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
2 (0xffff800000053040) kvm_wall_clock_ cpu0 waiting 3.95907e-15 8305 std::_Function_handler<void(), kvmclock::kvmclock()::<lambda()> >::_M_invoke(const std::_Any_data &) at /usr/include/c++/10/bits/std_function.h:291
3 (0xffff800000071040) page-access-sca cpu0 waiting 1.48546e-11 8301839 pagecache::access_scanner::run() at core/pagecache.cc:695
4 (0xffff800000089040) page_pool_l2 cpu0 waiting 2.02963e-16 11508850 memory::page_pool::l2::fill_thread() at core/mempool.cc:1440
5 (0xffff800000096040) itimer-real cpu? unstarted 0 0 ?? at arch/x64/entry.S:112
6 (0xffff800000122040) itimer-virt cpu? unstarted 0 0 ?? at arch/x64/entry.S:112
7 (0xffff800000159040) balancer0 cpu0 waiting 1.07872e-13 64790 ?? at arch/x64/entry.S:113
8 (0xffff800000171040) rcu0 cpu0 waiting 1.55096e-24 2804 osv::rcu::cpu_quiescent_state_thread::do_work() at core/rcu.cc:178
9 (0xffff80000018b040) page_pool_l1_0 cpu0 waiting 1.08365e-13 437879 memory::page_pool::l1::fill_thread() at core/mempool.cc:1352
10 (0xffff8000001a2040) percpu0 cpu0 waiting 4.3944e-25 786 workman::call_of_duty() at core/percpu-worker.cc:92
11 (0xffff8000001ba040) async_worker0 cpu0 waiting 1.01945e-21 3058 async::async_worker::run() at core/async.cc:158
12 (0xffff8000001d2040) idle0 cpu0 queued inf 536943227 std::_Function_handler<void(), sched::cpu::init_idle_thread()::<lambda()> >::_M_invoke(const std::_Any_data &) at /usr/include/c++/10/bits/std_function.h:291
13 (0xffff800000b22040) >init cpu0 waiting 5.84196e-24 484 osv::acpi_interrupt::process_interrupts() at drivers/acpi.cc:247
14 (0xffff800000b39040) >init cpu0 waiting 9.14373e-14 172890 std::_Function_handler<void(), sched::thread::reaper::reaper()::<lambda()> >::_M_invoke(const std::_Any_data &) at /usr/include/c++/10/bits/std_function.h:291
15 (0xffff800000b50040) thread taskq cpu0 waiting 6.15216e-24 1014 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
16 (0xffff800000b5b040) callout cpu0 waiting 9.43065e-14 117580 _callout_thread() at bsd/porting/callout.cc:138
17 (0xffff800000b74040) system_taskq_0 cpu0 waiting 5.77734e-24 321 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
18 (0xffff800000b7f040) system_taskq_1 cpu0 waiting 5.76441e-24 328 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
19 (0xffff800000b8a040) system_taskq_2 cpu0 waiting 5.73857e-24 257 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
20 (0xffff800000b95040) system_taskq_3 cpu0 waiting 5.76441e-24 310 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
21 (0xffff800000ba0040) system_taskq_4 cpu0 waiting 5.73857e-24 252 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
22 (0xffff800000bab040) system_taskq_5 cpu0 waiting 5.76441e-24 308 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
23 (0xffff800000bb6040) system_taskq_6 cpu0 waiting 5.75149e-24 290 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
24 (0xffff800000bc1040) system_taskq_7 cpu0 waiting 5.75149e-24 294 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
25 (0xffff80007fe41040) solthread-0x402 cpu0 waiting 3.34802e-16 26930 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
26 (0xffff80007fe4c040) solthread-0x402 cpu0 waiting 3.73453e-15 5615 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
27 (0xffff80007fe73040) netisr cpu0 waiting 1.78878e-23 3796 netisr_osv_thread_wrapper(void (*)(void*), void*) at bsd/sys/net/netisr1.cc:26
28 (0xffff800000bce040) >init cpu0 waiting 6.07803e-20 86062398 sched::thread::join() at core/sched.cc:1370
joining on <optimized out>
29 (0xffff800000be8040) virtio-net-rx cpu0 waiting 3.32942e-28 27785 virtio::virtio_driver::wait_for_queue(virtio::vring*, bool (virtio::vring::*)() const) at drivers/virtio.cc:158
30 (0xffff800000c10040) virtio-tx-0 cpu0 waiting 1.0469e-28 2239 osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::poll_until() at include/osv/percpu_xmit.hh:399
31 (0xffff800000d28040) virtio-blk cpu0 waiting 6.49072e-20 5057796 virtio::virtio_driver::wait_for_queue(virtio::vring*, bool (virtio::vring::*)() const) at drivers/virtio.cc:158
32 (0xffff800000d3f040) virtio-rng cpu0 waiting 3.37945e-13 798065 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
33 (0xffff800000d59040) isa-serial-inpu cpu0 waiting 1.28213e-23 4559 console::LineDiscipline::read_poll(console::console_driver*) at drivers/line-discipline.cc:86
34 (0xffff800000d70040) kbd-input cpu0 waiting 1.28988e-23 4650 console::LineDiscipline::read_poll(console::console_driver*) at drivers/line-discipline.cc:86
35 (0xffff800000d92040) rand_harvestq cpu0 waiting 1.17297e-12 3278500 synch_port::_msleep(void*, mtx*, int, char const*, int) at bsd/porting/synch.cc:101
36 (0xffff800000dac040) dhcp cpu0 waiting 6.64756e-22 734044 dhcp::dhcp_worker::dhcp_worker_fn() at core/dhcp.cc:828
37 (0xffff800000f01040) /usr/bin/java cpu0 waiting 1.8707e-20 13123899 sched::thread::join() at core/sched.cc:1370
joining on <optimized out>
38 (0xffff800001855040) >/usr/bin/java cpu0 waiting 2.00517e-13 131008482 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
39 (0xffff8000025ab040) VM Thread cpu0 waiting 3.94684e-13 574237 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
40 (0xffff8000027e3040) Reference Handl cpu0 waiting 2.10104e-21 193697 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
41 (0xffff8000028fb040) Finalizer cpu0 waiting 3.95082e-21 348549 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
42 (0xffff800002bf4040) Signal Dispatch cpu0 waiting 7.74241e-22 47401 semaphore::wait(unsigned int, sched::timer*) at core/semaphore.cc:46
43 (0xffff800002cfa040) Service Thread cpu0 waiting 2.41123e-21 168287 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
44 (0xffff800002e12040) C2 CompilerThre cpu0 waiting 2.21803e-18 30482642 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
45 (0xffff800002f3c040) C1 CompilerThre cpu0 waiting 8.66538e-19 31026992 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
46 (0xffff800003079040) Sweeper thread cpu0 waiting 8.82395e-22 52394 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
47 (0xffff800003774040) VM Periodic Tas cpu0 waiting 2.82989e-13 137938 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
48 (0xffff800003972040) Common-Cleaner cpu0 waiting 1.61176e-21 44635 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
49 (0xffff800003e3a040) SeedGenerator T cpu0 running 1.32268e-09 3735622772 clock_gettime(clockid_t, timespec*) at libc/time.cc:81
65 (0xffff8000009f5040) SeedGenerator T cpu0 waiting 2.62568e-13 12648 condvar::wait(lockfree::mutex*, sched::timer*) at core/condvar.cc:43
osv thread apply all bt
thread 0xffff800000016040 reclaimer
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...)
at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000016040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...)
at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x0, this=0xffff90000001dec0) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0x4090d850 <memory::reclaimer_thread+16>, user_mutex=user_mutex@entry=0x4090d900 <memory::free_page_ranges_lock>,
tmr=tmr@entry=0x0) at core/condvar.cc:43
#8 0x00000000403e617d in condvar::wait (tmr=0x0, user_mutex=..., this=0x4090d850 <memory::reclaimer_thread+16>) at include/osv/condvar.h:226
#9 memory::reclaimer::_do_reclaim (this=0x4090d840 <memory::reclaimer_thread>) at core/mempool.cc:1071
#10 0x00000000403e635c in operator() (__closure=<optimized out>, __closure=<optimized out>) at core/mempool.cc:1026
#11 std::__invoke_impl<void, memory::reclaimer::reclaimer()::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#12 std::__invoke_r<void, memory::reclaimer::reclaimer()::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#13 std::_Function_handler<void(), memory::reclaimer::reclaimer()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...)
at /usr/include/c++/10/bits/std_function.h:291
#14 0x00000000403fbffa in sched::thread::main (this=0xffff800000016040) at core/sched.cc:1267
#15 sched::thread_main_c (t=0xffff800000016040) at arch/x64/arch-switch.hh:325
#16 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000053040 kvm_wall_clock_
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...)
at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=<optimized out>) at core/sched.cc:1273
#4 0x00000000403fce56 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, sched::thread::sleep_impl(sched::timer&)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<sched::thread::sleep_impl(sched::timer&)::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 sched::thread::sleep_impl (t=...) at core/sched.cc:1429
#7 0x000000004038057d in sched::thread::sleep<long, std::ratio<1l, 1l> > (duration=...) at include/osv/sched.hh:1314
#8 operator() (__closure=0xffff800000053070) at drivers/kvmclock.cc:69
#9 std::__invoke_impl<void, kvmclock::kvmclock()::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#10 std::__invoke_r<void, kvmclock::kvmclock()::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
--Type <RET> for more, q to quit, c to continue without paging--c
#11 std::_Function_handler<void(), kvmclock::kvmclock()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#12 0x00000000403fbffa in sched::thread::main (this=0xffff800000053040) at core/sched.cc:1267
#13 sched::thread_main_c (t=0xffff800000053040) at arch/x64/arch-switch.hh:325
#14 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000071040 page-access-sca
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=<optimized out>) at core/sched.cc:1273
#4 0x00000000403fce56 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, sched::thread::sleep_impl(sched::timer&)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<sched::thread::sleep_impl(sched::timer&)::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 sched::thread::sleep_impl (t=...) at core/sched.cc:1429
#7 0x000000004038aee6 in sched::thread::sleep<long, std::ratio<1l, 1000000000l> > (duration=...) at include/osv/sched.hh:1314
#8 0x00000000403e30e6 in pagecache::access_scanner::run (this=0x4090c940 <pagecache::s_access_scanner>) at core/pagecache.cc:695
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000071040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000071040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000089040 page_pool_l2
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000089040) at core/sched.cc:1273
#4 0x00000000403e8a5c in sched::thread::do_wait_for<sched::thread::dummy_lock, sched::wait_object<memory::page_pool::l2::fill_thread()::<lambda()> > > (mtx=<synthetic pointer>...) at include/osv/sched.hh:1252
#5 sched::thread::wait_for<memory::page_pool::l2::fill_thread()::<lambda()> > () at include/osv/sched.hh:1276
#6 memory::page_pool::l2::fill_thread (this=0x4090d640 <memory::page_pool::global_l2>) at core/mempool.cc:1440
#7 0x00000000403eb80c in memory::page_pool::l2::l2()::{lambda()#1}::operator()() const (__closure=<optimized out>, __closure=<optimized out>) at core/mempool.cc:1247
#8 std::__invoke_impl<void, memory::page_pool::l2::l2()::{lambda()#1}&>(std::__invoke_other, memory::page_pool::l2::l2()::{lambda()#1}&) (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#9 std::__invoke_r<void, memory::page_pool::l2::l2()::{lambda()#1}&>(memory::page_pool::l2::l2()::{lambda()#1}&) (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#10 std::_Function_handler<void (), memory::page_pool::l2::l2()::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#11 0x00000000403fbffa in sched::thread::main (this=0xffff800000089040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff800000089040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000096040 itimer-real
#0 thread_main () at arch/x64/entry.S:112
thread 0xffff800000122040 itimer-virt
#0 thread_main () at arch/x64/entry.S:112
thread 0xffff800000159040 balancer0
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=<optimized out>) at core/sched.cc:1273
#4 0x00000000403fc76d in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, sched::cpu::load_balance()::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<sched::cpu::load_balance()::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 sched::cpu::load_balance (this=0xffff80000001e040) at core/sched.cc:730
#7 0x00000000403fbffa in sched::thread::main (this=0xffff800000159040) at core/sched.cc:1267
#8 sched::thread_main_c (t=0xffff800000159040) at arch/x64/arch-switch.hh:325
#9 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000171040 rcu0
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff80000018b040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000171040) at core/sched.cc:1273
#4 0x00000000403de6fe in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, osv::rcu::cpu_quiescent_state_thread::do_work()::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<osv::rcu::cpu_quiescent_state_thread::do_work()::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 osv::rcu::cpu_quiescent_state_thread::do_work (this=0xffffa0000015f7a0) at core/rcu.cc:178
#7 0x00000000403dea49 in osv::rcu::cpu_quiescent_state_thread::work (this=<optimized out>) at core/rcu.cc:113
#8 0x00000000403dea6c in operator() (__closure=<optimized out>, __closure=<optimized out>) at core/rcu.cc:67
#9 std::__invoke_impl<void, osv::rcu::cpu_quiescent_state_thread::cpu_quiescent_state_thread(sched::cpu*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#10 std::__invoke_r<void, osv::rcu::cpu_quiescent_state_thread::cpu_quiescent_state_thread(sched::cpu*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#11 std::_Function_handler<void(), osv::rcu::cpu_quiescent_state_thread::cpu_quiescent_state_thread(sched::cpu*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#12 0x00000000403fbffa in sched::thread::main (this=0xffff800000171040) at core/sched.cc:1267
#13 sched::thread_main_c (t=0xffff800000171040) at arch/x64/arch-switch.hh:325
#14 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff80000018b040 page_pool_l1_0
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff80000018b040) at core/sched.cc:1273
#4 0x00000000403e88a1 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, memory::page_pool::l1::fill_thread()::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<memory::page_pool::l1::fill_thread()::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 memory::page_pool::l1::fill_thread () at core/mempool.cc:1352
#7 0x00000000403eb7f9 in memory::page_pool::l1::l1(sched::cpu*)::{lambda()#1}::operator()() const (__closure=<optimized out>) at core/mempool.cc:1164
#8 std::__invoke_impl<void, memory::page_pool::l1::l1(sched::cpu*)::{lambda()#1}&>(std::__invoke_other, memory::page_pool::l1::l1(sched::cpu*)::{lambda()#1}&) (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#9 std::__invoke_r<void, memory::page_pool::l1::l1(sched::cpu*)::{lambda()#1}&>(memory::page_pool::l1::l1(sched::cpu*)::{lambda()#1}&) (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#10 std::_Function_handler<void (), memory::page_pool::l1::l1(sched::cpu*)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#11 0x00000000403fbffa in sched::thread::main (this=0xffff80000018b040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff80000018b040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff8000001a2040 percpu0
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff8000001ba040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff8000001a2040) at core/sched.cc:1273
#4 0x000000004042084f in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, workman::call_of_duty()::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<workman::call_of_duty()::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 workman::call_of_duty () at core/percpu-worker.cc:92
#7 0x0000000040420a19 in operator() (__closure=<optimized out>) at core/percpu-worker.cc:124
#8 std::__invoke_impl<void, workman::pcpu_init()::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#9 std::__invoke_r<void, workman::pcpu_init()::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#10 std::_Function_handler<void(), workman::pcpu_init()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#11 0x00000000403fbffa in sched::thread::main (this=0xffff8000001a2040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff8000001a2040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff8000001ba040 async_worker0
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800001855040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff8000001ba040) at core/sched.cc:1273
#4 0x0000000040429564 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, async::async_worker::run()::{lambda()#1}>(sched::thread::dummy_lock&, async::async_worker::run()::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<async::async_worker::run()::{lambda()#1}>(async::async_worker::run()::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 async::async_worker::run (this=0xffffa000001a87c0) at core/async.cc:158
#7 0x00000000403fbffa in sched::thread::main (this=0xffff8000001ba040) at core/sched.cc:1267
#8 sched::thread_main_c (t=0xffff8000001ba040) at arch/x64/arch-switch.hh:325
#9 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff8000001d2040 idle0
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800000d28040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fc2b4 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::cpu::idle (this=0xffff80000001e040) at core/sched.cc:474
#4 0x00000000403fc2dc in operator() (__closure=<optimized out>, __closure=<optimized out>) at core/sched.cc:165
#5 std::__invoke_impl<void, sched::cpu::init_idle_thread()::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#6 std::__invoke_r<void, sched::cpu::init_idle_thread()::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#7 std::_Function_handler<void(), sched::cpu::init_idle_thread()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#8 0x00000000403fbffa in sched::thread::main (this=0xffff8000001d2040) at core/sched.cc:1267
#9 sched::thread_main_c (t=0xffff8000001d2040) at arch/x64/arch-switch.hh:325
#10 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000b22040 >init
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000b22040) at core/sched.cc:1273
#4 0x0000000040381de0 in sched::thread::do_wait_for<sched::thread::dummy_lock, sched::wait_object<osv::acpi_interrupt::process_interrupts()::{lambda()#1}>, sched::wait_object<osv::acpi_interrupt::process_interrupts()::{lambda()#2}> >(sched::thread::dummy_lock&, sched::wait_object<osv::acpi_interrupt::process_interrupts()::{lambda()#1}>&&, sched::wait_object<osv::acpi_interrupt::process_interrupts()::{lambda()#2}>&&) (mtx=<synthetic pointer>...) at include/osv/sched.hh:1252
#5 sched::thread::wait_for<osv::acpi_interrupt::process_interrupts()::{lambda()#1}, osv::acpi_interrupt::process_interrupts()::{lambda()#2}>(osv::acpi_interrupt::process_interrupts()::{lambda()#1}&&, osv::acpi_interrupt::process_interrupts()::{lambda()#2}&&) () at include/osv/sched.hh:1276
#6 osv::acpi_interrupt::process_interrupts (this=0xffffa0000097f380) at drivers/acpi.cc:247
#7 0x00000000403fbffa in sched::thread::main (this=0xffff800000b22040) at core/sched.cc:1267
#8 sched::thread_main_c (t=0xffff800000b22040) at arch/x64/arch-switch.hh:325
#9 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000b39040 >init
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff80000018b040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=<optimized out>) at core/sched.cc:1273
#4 0x00000000403fd180 in sched::thread::do_wait_until<sched::noninterruptible, lockfree::mutex, sched::thread::reaper::reap()::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<sched::thread::reaper::reap()::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1132
#6 sched::thread::reaper::reap (this=0xffffa00000978400) at core/sched.cc:1612
#7 0x00000000403fd29c in operator() (__closure=<optimized out>, __closure=<optimized out>) at core/sched.cc:1603
#8 std::__invoke_impl<void, sched::thread::reaper::reaper()::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#9 std::__invoke_r<void, sched::thread::reaper::reaper()::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#10 std::_Function_handler<void(), sched::thread::reaper::reaper()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#11 0x00000000403fbffa in sched::thread::main (this=0xffff800000b39040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff800000b39040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000b50040 thread taskq
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000b50040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa00000978530, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa00000978530, p=0xffffa00000978500, tq=0xffffa00000978500) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000b50040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000b50040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000b5b040 callout
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000b5b040) at core/sched.cc:1273
#4 0x0000000040258eef in sched::thread::do_wait_until<sched::noninterruptible, lockfree::mutex, _callout_thread()::<lambda()> > (mtx=..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<_callout_thread()::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1132
#6 _callout_thread () at bsd/porting/callout.cc:138
#7 0x00000000403fbffa in sched::thread::main (this=0xffff800000b5b040) at core/sched.cc:1267
#8 sched::thread_main_c (t=0xffff800000b5b040) at arch/x64/arch-switch.hh:325
#9 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000b74040 system_taskq_0
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000b74040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa000009786b0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa000009786b0, p=0xffffa00000978680, tq=0xffffa00000978680) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000b74040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000b74040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000b7f040 system_taskq_1
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000b7f040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa000009786b0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa000009786b0, p=0xffffa00000978680, tq=0xffffa00000978680) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000b7f040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000b7f040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000b8a040 system_taskq_2
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000b8a040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa000009786b0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa000009786b0, p=0xffffa00000978680, tq=0xffffa00000978680) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000b8a040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000b8a040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000b95040 system_taskq_3
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000b95040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa000009786b0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa000009786b0, p=0xffffa00000978680, tq=0xffffa00000978680) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000b95040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000b95040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000ba0040 system_taskq_4
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000ba0040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa000009786b0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa000009786b0, p=0xffffa00000978680, tq=0xffffa00000978680) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000ba0040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000ba0040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000bab040 system_taskq_5
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000bab040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa000009786b0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa000009786b0, p=0xffffa00000978680, tq=0xffffa00000978680) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000bab040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000bab040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000bb6040 system_taskq_6
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000bb6040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa000009786b0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa000009786b0, p=0xffffa00000978680, tq=0xffffa00000978680) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000bb6040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000bb6040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000bc1040 system_taskq_7
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0x408b1df0) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000bc1040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0xffffa000009786b0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=0) at bsd/porting/synch.cc:101
#7 0x00000000402542c5 in TQ_SLEEP (t=0, wm=0x4068a6ea "-", pri=0, m=0xffffa000009786b0, p=0xffffa00000978680, tq=0xffffa00000978680) at bsd/sys/kern/subr_taskqueue.c:92
#8 taskqueue_thread_loop (arg=<optimized out>) at bsd/sys/kern/subr_taskqueue.c:474
#9 0x00000000403fbffa in sched::thread::main (this=0xffff800000bc1040) at core/sched.cc:1267
#10 sched::thread_main_c (t=0xffff800000bc1040) at arch/x64/arch-switch.hh:325
#11 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff80007fe41040 solthread-0x402
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff80007fe41040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0xffff80007fe4aea0, this=0xffff80007fe4ae00) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0x408f0520 <arc_reclaim_thr_cv>, user_mutex=user_mutex@entry=0x408f0560 <arc_reclaim_thr_lock>, tmr=tmr@entry=0xffff80007fe4aea0) at core/condvar.cc:43
#8 0x0000000040256cad in condvar::wait<long, std::ratio<1l, 1000000000l> > (duration=..., user_mutex=0x408f0560 <arc_reclaim_thr_lock>, this=0x408f0520 <arc_reclaim_thr_cv>) at include/osv/condvar.h:151
#9 cv_timedwait (cv=cv@entry=0x408f0520 <arc_reclaim_thr_cv>, mutex=mutex@entry=0x408f0560 <arc_reclaim_thr_lock>, tmo=tmo@entry=1000) at bsd/porting/netport1.cc:66
#10 0x00000000402c427e in arc_reclaim_thread (dummy=<optimized out>) at bsd/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2539
#11 0x00000000403fbffa in sched::thread::main (this=0xffff80007fe41040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff80007fe41040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff80007fe4c040 solthread-0x402
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff80007fe41040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff80007fe4c040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0xffff80007fe55e30, this=0xffff80007fe55d80) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0x408e05c0 <l2arc_feed_thr_cv>, user_mutex=user_mutex@entry=0x408e0600 <l2arc_feed_thr_lock>, tmr=tmr@entry=0xffff80007fe55e30) at core/condvar.cc:43
#8 0x0000000040256cad in condvar::wait<long, std::ratio<1l, 1000000000l> > (duration=..., user_mutex=0x408e0600 <l2arc_feed_thr_lock>, this=0x408e05c0 <l2arc_feed_thr_cv>) at include/osv/condvar.h:151
#9 cv_timedwait (cv=cv@entry=0x408e05c0 <l2arc_feed_thr_cv>, mutex=mutex@entry=0x408e0600 <l2arc_feed_thr_lock>, tmo=tmo@entry=1000) at bsd/porting/netport1.cc:66
#10 0x00000000402c0e88 in l2arc_feed_thread (dummy=<optimized out>) at bsd/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:4983
#11 0x00000000403fbffa in sched::thread::main (this=0xffff80007fe4c040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff80007fe4c040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff80007fe73040 netisr
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800000dac040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff80007fe73040) at core/sched.cc:1273
#4 0x0000000040268c7a in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, netisr_osv_thread_wrapper(netisr_osv_handler_t, void*)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<netisr_osv_thread_wrapper(netisr_osv_handler_t, void*)::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 netisr_osv_thread_wrapper (handler=0x402678f0 <swi_net(void*)>, arg=0x408b92e0 <main_nws>) at bsd/sys/net/netisr1.cc:26
#7 0x0000000040268d10 in operator() (__closure=<optimized out>, __closure=<optimized out>) at bsd/sys/net/netisr1.cc:39
#8 std::__invoke_impl<void, netisr_osv_start_thread(netisr_osv_handler_t, void*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#9 std::__invoke_r<void, netisr_osv_start_thread(netisr_osv_handler_t, void*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#10 std::_Function_handler<void(), netisr_osv_start_thread(netisr_osv_handler_t, void*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#11 0x00000000403fbffa in sched::thread::main (this=0xffff80007fe73040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff80007fe73040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000bce040 >init
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff8000001d2040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=<optimized out>) at core/sched.cc:1273
#4 0x00000000403fd09f in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, sched::thread::join()::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<sched::thread::join()::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 sched::thread::join (this=<optimized out>) at core/sched.cc:1370
#7 0x0000000040460416 in pthread_private::pthread::join (retval=0x0, this=0xffffa000009a45c0) at /usr/include/c++/10/bits/unique_ptr.h:173
#8 pthread_join (thread=18446638520603395520, retval=0x0) at libc/pthread.cc:246
#9 0x000000004042b198 in osv::application::join (this=0xffffa0007ffb9e10) at core/app.cc:259
#10 0x000000004022bd7f in do_main_thread (_main_args=<optimized out>) at /usr/include/c++/10/bits/shared_ptr_base.h:1324
#11 0x000000004045f6a6 in operator() (__closure=0xffffa0007ff77100) at libc/pthread.cc:115
#12 std::__invoke_impl<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#13 std::__invoke_r<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#14 std::_Function_handler<void(), pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#15 0x00000000403fbffa in sched::thread::main (this=0xffff800000bce040) at core/sched.cc:1267
#16 sched::thread_main_c (t=0xffff800000bce040) at arch/x64/arch-switch.hh:325
#17 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000be8040 virtio-net-rx
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800000dac040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000be8040) at core/sched.cc:1273
#4 0x000000004036c591 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, virtio::virtio_driver::wait_for_queue(virtio::vring*, bool (virtio::vring::*)() const)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<virtio::virtio_driver::wait_for_queue(virtio::vring*, bool (virtio::vring::*)() const)::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 virtio::virtio_driver::wait_for_queue (this=this@entry=0xffff90007ff82000, queue=queue@entry=0xffffa0007ff77300, pred=<optimized out>) at drivers/virtio.cc:158
#7 0x00000000403715d2 in virtio::net::receiver (this=0xffff90007ff82000) at drivers/virtio-net.cc:468
#8 0x0000000040371b5c in operator() (__closure=<optimized out>, __closure=<optimized out>) at drivers/virtio-net.cc:242
#9 std::__invoke_impl<void, virtio::net::net(virtio::virtio_device&)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#10 std::__invoke_r<void, virtio::net::net(virtio::virtio_device&)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#11 std::_Function_handler<void(), virtio::net::net(virtio::virtio_device&)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#12 0x00000000403fbffa in sched::thread::main (this=0xffff800000be8040) at core/sched.cc:1267
#13 sched::thread_main_c (t=0xffff800000be8040) at arch/x64/arch-switch.hh:325
#14 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000c10040 virtio-tx-0
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800000bce040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000c10040) at core/sched.cc:1273
#4 0x00000000403745cb in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::poll_until()::{lambda()#1}>(sched::thread::dummy_lock&, osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::poll_until()::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::poll_until()::{lambda()#1}>(osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::poll_until()::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::poll_until() (this=0xffff90007ff823c0) at include/osv/percpu_xmit.hh:399
#7 0x000000004037471c in osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::xmitter(virtio::net::txq*, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda()#1}::operator()() const (this=<optimized out>, this=<optimized out>) at include/osv/percpu_xmit.hh:222
#8 std::__invoke_impl<void, osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::xmitter(virtio::net::txq*, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda()#1}&>(std::__invoke_other, osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::xmitter(virtio::net::txq*, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda()#1}&) (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#9 std::__invoke_r<void, osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::xmitter(virtio::net::txq*, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda()#1}&>(osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::xmitter(virtio::net::txq*, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda()#1}&) (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#10 std::_Function_handler<void (), osv::xmitter<virtio::net::txq, 4096u, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> > >::xmitter(virtio::net::txq*, std::function<bool ()>, boost::iterators::function_output_iterator<osv::xmitter_functor<virtio::net::txq> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#11 0x00000000403fbffa in sched::thread::main (this=0xffff800000c10040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff800000c10040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000d28040 virtio-blk
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800002e12040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000d28040) at core/sched.cc:1273
#4 0x000000004036c591 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, virtio::virtio_driver::wait_for_queue(virtio::vring*, bool (virtio::vring::*)() const)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<virtio::virtio_driver::wait_for_queue(virtio::vring*, bool (virtio::vring::*)() const)::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 virtio::virtio_driver::wait_for_queue (this=this@entry=0xffffa00000cff400, queue=queue@entry=0xffffa0007ff77700, pred=<optimized out>) at drivers/virtio.cc:158
#7 0x000000004037b00a in virtio::blk::req_done (this=0xffffa00000cff400) at drivers/virtio-blk.cc:235
#8 0x000000004037b29c in operator() (__closure=<optimized out>, __closure=<optimized out>) at drivers/virtio-blk.cc:130
#9 std::__invoke_impl<void, virtio::blk::blk(virtio::virtio_device&)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#10 std::__invoke_r<void, virtio::blk::blk(virtio::virtio_device&)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#11 std::_Function_handler<void(), virtio::blk::blk(virtio::virtio_device&)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#12 0x00000000403fbffa in sched::thread::main (this=0xffff800000d28040) at core/sched.cc:1267
#13 sched::thread_main_c (t=0xffff800000d28040) at arch/x64/arch-switch.hh:325
#14 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000d3f040 virtio-rng
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000d3f040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x0, this=0xffff800000d54f40) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0xffffa00000cffa48, user_mutex=user_mutex@entry=0xffffa00000cffad0, tmr=tmr@entry=0x0) at core/condvar.cc:43
#8 0x000000004037ed85 in condvar::wait_until<virtio::rng::worker()::<lambda()> > (pred=..., mtx=..., this=0xffffa00000cffa48) at include/osv/condvar.h:237
#9 virtio::rng::worker (this=0xffffa00000cff800) at drivers/virtio-rng.cc:104
#10 0x000000004037edcc in operator() (__closure=<optimized out>, __closure=<optimized out>) at drivers/virtio-rng.cc:41
#11 std::__invoke_impl<void, virtio::rng::rng(virtio::virtio_device&)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#12 std::__invoke_r<void, virtio::rng::rng(virtio::virtio_device&)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#13 std::_Function_handler<void(), virtio::rng::rng(virtio::virtio_device&)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#14 0x00000000403fbffa in sched::thread::main (this=0xffff800000d3f040) at core/sched.cc:1267
#15 sched::thread_main_c (t=0xffff800000d3f040) at arch/x64/arch-switch.hh:325
#16 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000d59040 isa-serial-inpu
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800000bce040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000d59040) at core/sched.cc:1273
#4 0x0000000040350de0 in sched::thread::do_wait_until<sched::noninterruptible, lockfree::mutex, console::LineDiscipline::read_poll(console::console_driver*)::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<console::LineDiscipline::read_poll(console::console_driver*)::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1132
#6 console::LineDiscipline::read_poll (this=0xffffa0007ff77900, driver=0x408f5740 <console::arch_early_console>) at drivers/line-discipline.cc:86
#7 0x00000000403fbffa in sched::thread::main (this=0xffff800000d59040) at core/sched.cc:1267
#8 sched::thread_main_c (t=0xffff800000d59040) at arch/x64/arch-switch.hh:325
#9 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000d70040 kbd-input
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800000bce040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000d70040) at core/sched.cc:1273
#4 0x0000000040350de0 in sched::thread::do_wait_until<sched::noninterruptible, lockfree::mutex, console::LineDiscipline::read_poll(console::console_driver*)::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<console::LineDiscipline::read_poll(console::console_driver*)::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1132
#6 console::LineDiscipline::read_poll (this=0xffffa0007ff77900, driver=0xffff800000bd4040) at drivers/line-discipline.cc:86
#7 0x00000000403fbffa in sched::thread::main (this=0xffff800000d70040) at core/sched.cc:1267
#8 sched::thread_main_c (t=0xffff800000d70040) at arch/x64/arch-switch.hh:325
#9 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000d92040 rand_harvestq
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800000d3f040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000d92040) at core/sched.cc:1273
#4 0x0000000040259e94 in sched::thread::do_wait_until<sched::interruptible, sched::thread::dummy_lock, synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until_interruptible<synch_port::_msleep(void*, mtx*, int, char const*, int)::<lambda()> > (pred=...) at include/osv/sched.hh:1147
#6 synch_port::_msleep (this=<optimized out>, chan=<optimized out>, mtx=0x0, priority=<optimized out>, wmesg=<optimized out>, timo_hz=100) at bsd/porting/synch.cc:101
#7 0x00000000402a9c25 in random_kthread (arg=0x402a9250 <random_process_event(harvest*)>) at bsd/sys/dev/random/random_harvestq.cc:112
#8 0x00000000403fbffa in sched::thread::main (this=0xffff800000d92040) at core/sched.cc:1267
#9 sched::thread_main_c (t=0xffff800000d92040) at arch/x64/arch-switch.hh:325
#10 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000dac040 dhcp
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800000071040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800000dac040) at core/sched.cc:1273
#4 0x0000000040423f90 in sched::thread::do_wait_until<sched::noninterruptible, lockfree::mutex, dhcp::dhcp_worker::dhcp_worker_fn()::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<dhcp::dhcp_worker::dhcp_worker_fn()::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1132
#6 dhcp::dhcp_worker::dhcp_worker_fn (this=0x40911de0 <net_dhcp_worker>) at core/dhcp.cc:828
#7 0x00000000404240dc in operator() (__closure=<optimized out>, __closure=<optimized out>) at core/dhcp.cc:773
#8 std::__invoke_impl<void, dhcp::dhcp_worker::init()::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#9 std::__invoke_r<void, dhcp::dhcp_worker::init()::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#10 std::_Function_handler<void(), dhcp::dhcp_worker::init()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#11 0x00000000403fbffa in sched::thread::main (this=0xffff800000dac040) at core/sched.cc:1267
#12 sched::thread_main_c (t=0xffff800000dac040) at arch/x64/arch-switch.hh:325
#13 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800000f01040 /usr/bin/java
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff8000001d2040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=<optimized out>) at core/sched.cc:1273
#4 0x00000000403fd09f in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, sched::thread::join()::<lambda()> > (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<sched::thread::join()::<lambda()> > (pred=...) at include/osv/sched.hh:1125
#6 sched::thread::join (this=<optimized out>) at core/sched.cc:1370
#7 0x0000000040460416 in pthread_private::pthread::join (retval=0x2000006fcbd8, this=0xffffa000009a46f0) at /usr/include/c++/10/bits/unique_ptr.h:173
#8 pthread_join (thread=18446638520603395824, retval=0x2000006fcbd8) at libc/pthread.cc:246
#9 0x0000100000014d3d in ContinueInNewThread0 ()
#10 0x000010000001105e in ContinueInNewThread ()
#11 0x00001000000129fa in JLI_Launch ()
#12 0x00001000000062c3 in main ()
#13 0x000000004042d70d in osv::application::run_main (this=0xffffa0007ffb9e10) at core/app.cc:413
#14 0x000000004042d851 in __libc_start_main (main=0x100000006190 <main>) at core/app.cc:37
#15 0x000010000000636e in _start ()
thread 0xffff800001855040 >/usr/bin/java
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800001855040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x0, this=0x2000007ffb00) at include/osv/wait_record.hh:46
#7 condvar::wait (this=0xffffa000014a9d80, user_mutex=0xffffa00003908d00, tmr=0x0) at core/condvar.cc:43
#8 0x0000100000c9036b in os::PlatformEvent::park() ()
#9 0x0000100000c5de25 in ObjectMonitor::wait(long, bool, Thread*) ()
#10 0x0000100000e676e8 in ObjectSynchronizer::wait(Handle, long, Thread*) ()
#11 0x00001000009a182f in JVM_MonitorWait ()
#12 0x000020000920fe17 in ?? ()
#13 0x0000000000000000 in ?? ()
thread 0xffff8000025ab040 VM Thread
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff8000025ab040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x200018d00a90, this=0x200018d00a00) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0xffffa0000179b300, user_mutex=0xffffa000017d4dc0, tmr=tmr@entry=0x200018d00a90) at core/condvar.cc:43
#8 0x000000004046110d in pthread_cond_timedwait(pthread_cond_t * __restrict__, pthread_mutex_t * __restrict__, const timespec * __restrict__) (cond=0xffffa00001755d50, mutex=0xffffa00001755d28, ts=<optimized out>) at libc/pthread.cc:623
#9 0x0000100000c8d85e in os::PlatformEvent::park(long) [clone .part.0] ()
#10 0x0000100000c38529 in Monitor::IWait(Thread*, long) ()
#11 0x0000100000c38f41 in Monitor::wait(bool, long, bool) ()
#12 0x0000100000f206a2 in VMThread::loop() ()
#13 0x0000100000f20e35 in VMThread::run() ()
#14 0x0000100000ea75bf in Thread::call_run() ()
#15 0x0000100000c863b6 in thread_native_entry(Thread*) ()
#16 0x000000004045f6a6 in operator() (__closure=0xffffa0000179c800) at libc/pthread.cc:115
#17 std::__invoke_impl<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#18 std::__invoke_r<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#19 std::_Function_handler<void(), pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#20 0x00000000403fbffa in sched::thread::main (this=0xffff8000025ab040) at core/sched.cc:1267
#21 sched::thread_main_c (t=0xffff8000025ab040) at arch/x64/arch-switch.hh:325
#22 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff8000027e3040 Reference Handl
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800001855040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff8000027e3040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x0, this=0x200018e00800) at include/osv/wait_record.hh:46
#7 condvar::wait (this=0xffffa0000179b780, user_mutex=0xffffa00002567300, tmr=0x0) at core/condvar.cc:43
#8 0x0000100000c9036b in os::PlatformEvent::park() ()
#9 0x0000100000c38468 in Monitor::IWait(Thread*, long) ()
#10 0x0000100000c38fbe in Monitor::wait(bool, long, bool) ()
#11 0x00001000009aa886 in JVM_WaitForReferencePendingList ()
#12 0x000020000920fe17 in ?? ()
#13 0x000020000920fad6 in ?? ()
#14 0x0000200018e00988 in ?? ()
#15 0x0000000800514038 in ?? ()
#16 0x0000200018e009e8 in ?? ()
#17 0x0000000800009cd8 in ?? ()
#18 0x0000000000000000 in ?? ()
thread 0xffff8000028fb040 Finalizer
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800001855040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff8000028fb040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x0, this=0x200018f00640) at include/osv/wait_record.hh:46
#7 condvar::wait (this=0xffffa0000179bc00, user_mutex=0xffffa00002567cc0, tmr=0x0) at core/condvar.cc:43
#8 0x0000100000c9036b in os::PlatformEvent::park() ()
#9 0x0000100000c5de25 in ObjectMonitor::wait(long, bool, Thread*) ()
#10 0x0000100000e676e8 in ObjectSynchronizer::wait(Handle, long, Thread*) ()
#11 0x00001000009a182f in JVM_MonitorWait ()
#12 0x000020000920fe17 in ?? ()
#13 0x0000200018f00920 in ?? ()
#14 0x00000000000000b6 in ?? ()
#15 0xffff900002561000 in ?? ()
#16 0x0000200018f008c8 in ?? ()
#17 0x0000000000000000 in ?? ()
thread 0xffff800002bf4040 Signal Dispatch
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800001855040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800002bf4040) at core/sched.cc:1273
#4 0x00000000403db737 in sched::thread::do_wait_until<sched::noninterruptible, lockfree::mutex, semaphore::wait(unsigned int, sched::timer*)::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<semaphore::wait(unsigned int, sched::timer*)::<lambda()> > (pred=..., mtx=...) at include/osv/sched.hh:1132
#6 semaphore::wait (this=0xffffa00001473180, units=units@entry=1, tmr=tmr@entry=0x0) at core/semaphore.cc:46
#7 0x0000000040466463 in sem_wait (s=<optimized out>) at libc/sem.cc:47
#8 0x0000100000d4a2da in PosixSemaphore::wait() ()
#9 0x0000100000c807c0 in os::signal_wait() ()
#10 0x0000100000c73165 in signal_thread_entry(JavaThread*, Thread*) ()
#11 0x0000100000eac308 in JavaThread::thread_main_inner() ()
#12 0x0000100000ea75bf in Thread::call_run() ()
#13 0x0000100000c863b6 in thread_native_entry(Thread*) ()
#14 0x000000004045f6a6 in operator() (__closure=0xffffa00002798b00) at libc/pthread.cc:115
#15 std::__invoke_impl<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#16 std::__invoke_r<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#17 std::_Function_handler<void(), pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#18 0x00000000403fbffa in sched::thread::main (this=0xffff800002bf4040) at core/sched.cc:1267
#19 sched::thread_main_c (t=0xffff800002bf4040) at arch/x64/arch-switch.hh:325
#20 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800002cfa040 Service Thread
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800001855040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800002cfa040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x0, this=0x200019100c40) at include/osv/wait_record.hh:46
#7 condvar::wait (this=0xffffa000028ac880, user_mutex=0xffffa000028107c0, tmr=0x0) at core/condvar.cc:43
#8 0x0000100000c9036b in os::PlatformEvent::park() ()
#9 0x0000100000c38468 in Monitor::IWait(Thread*, long) ()
#10 0x0000100000c38f41 in Monitor::wait(bool, long, bool) ()
#11 0x0000100000d4ab42 in ServiceThread::service_thread_entry(JavaThread*, Thread*) ()
#12 0x0000100000eac308 in JavaThread::thread_main_inner() ()
#13 0x0000100000ea75bf in Thread::call_run() ()
#14 0x0000100000c863b6 in thread_native_entry(Thread*) ()
#15 0x000000004045f6a6 in operator() (__closure=0xffffa00002b03100) at libc/pthread.cc:115
#16 std::__invoke_impl<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#17 std::__invoke_r<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#18 std::_Function_handler<void(), pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#19 0x00000000403fbffa in sched::thread::main (this=0xffff800002cfa040) at core/sched.cc:1267
#20 sched::thread_main_c (t=0xffff800002cfa040) at arch/x64/arch-switch.hh:325
#21 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800002e12040 C2 CompilerThre
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800002e12040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x200019200b90, this=0x200019200b00) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0xffffa000028acf00, user_mutex=0xffffa00002810c40, tmr=tmr@entry=0x200019200b90) at core/condvar.cc:43
#8 0x000000004046110d in pthread_cond_timedwait(pthread_cond_t * __restrict__, pthread_mutex_t * __restrict__, const timespec * __restrict__) (cond=0xffffa000029f7750, mutex=0xffffa000029f7728, ts=<optimized out>) at libc/pthread.cc:623
#9 0x0000100000c8d85e in os::PlatformEvent::park(long) [clone .part.0] ()
#10 0x0000100000c38529 in Monitor::IWait(Thread*, long) ()
#11 0x0000100000c38fbe in Monitor::wait(bool, long, bool) ()
#12 0x00001000006a0341 in CompileQueue::get() ()
#13 0x00001000006a375f in CompileBroker::compiler_thread_loop() ()
#14 0x0000100000eac308 in JavaThread::thread_main_inner() ()
#15 0x0000100000ea75bf in Thread::call_run() ()
#16 0x0000100000c863b6 in thread_native_entry(Thread*) ()
#17 0x000000004045f6a6 in operator() (__closure=0xffffa00002b03700) at libc/pthread.cc:115
#18 std::__invoke_impl<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#19 std::__invoke_r<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#20 std::_Function_handler<void(), pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#21 0x00000000403fbffa in sched::thread::main (this=0xffff800002e12040) at core/sched.cc:1267
#22 sched::thread_main_c (t=0xffff800002e12040) at arch/x64/arch-switch.hh:325
#23 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800002f3c040 C1 CompilerThre
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800002f3c040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x200019300b90, this=0x200019300b00) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0xffffa000029f8500, user_mutex=0xffffa00002d1d380, tmr=tmr@entry=0x200019300b90) at core/condvar.cc:43
#8 0x000000004046110d in pthread_cond_timedwait(pthread_cond_t * __restrict__, pthread_mutex_t * __restrict__, const timespec * __restrict__) (cond=0xffffa00002b00550, mutex=0xffffa00002b00528, ts=<optimized out>) at libc/pthread.cc:623
#9 0x0000100000c8d85e in os::PlatformEvent::park(long) [clone .part.0] ()
#10 0x0000100000c38529 in Monitor::IWait(Thread*, long) ()
#11 0x0000100000c38fbe in Monitor::wait(bool, long, bool) ()
#12 0x00001000006a0341 in CompileQueue::get() ()
#13 0x00001000006a375f in CompileBroker::compiler_thread_loop() ()
#14 0x0000100000eac308 in JavaThread::thread_main_inner() ()
#15 0x0000100000ea75bf in Thread::call_run() ()
#16 0x0000100000c863b6 in thread_native_entry(Thread*) ()
#17 0x000000004045f6a6 in operator() (__closure=0xffffa00002b03e00) at libc/pthread.cc:115
#18 std::__invoke_impl<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#19 std::__invoke_r<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#20 std::_Function_handler<void(), pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#21 0x00000000403fbffa in sched::thread::main (this=0xffff800002f3c040) at core/sched.cc:1267
#22 sched::thread_main_c (t=0xffff800002f3c040) at arch/x64/arch-switch.hh:325
#23 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800003079040 Sweeper thread
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800002f3c040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800003079040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x200019400cb0, this=0x200019400c00) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0xffffa000029f8a00, user_mutex=0xffffa00002e28140, tmr=tmr@entry=0x200019400cb0) at core/condvar.cc:43
#8 0x000000004046110d in pthread_cond_timedwait(pthread_cond_t * __restrict__, pthread_mutex_t * __restrict__, const timespec * __restrict__) (cond=0xffffa00002b00f50, mutex=0xffffa00002b00f28, ts=<optimized out>) at libc/pthread.cc:623
#9 0x0000100000c8d85e in os::PlatformEvent::park(long) [clone .part.0] ()
#10 0x0000100000c38529 in Monitor::IWait(Thread*, long) ()
#11 0x0000100000c38f41 in Monitor::wait(bool, long, bool) ()
#12 0x0000100000e5f450 in NMethodSweeper::sweeper_loop() ()
#13 0x0000100000eac308 in JavaThread::thread_main_inner() ()
#14 0x0000100000ea75bf in Thread::call_run() ()
#15 0x0000100000c863b6 in thread_native_entry(Thread*) ()
#16 0x000000004045f6a6 in operator() (__closure=0xffffa00002d18400) at libc/pthread.cc:115
#17 std::__invoke_impl<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#18 std::__invoke_r<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#19 std::_Function_handler<void(), pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#20 0x00000000403fbffa in sched::thread::main (this=0xffff800003079040) at core/sched.cc:1267
#21 sched::thread_main_c (t=0xffff800003079040) at arch/x64/arch-switch.hh:325
#22 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800003774040 VM Periodic Tas
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800003e3a040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800003774040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x200019501cd0, this=0x200019501c40) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0xffffa00002f53200, user_mutex=0xffffa00002f56880, tmr=tmr@entry=0x200019501cd0) at core/condvar.cc:43
#8 0x000000004046110d in pthread_cond_timedwait(pthread_cond_t * __restrict__, pthread_mutex_t * __restrict__, const timespec * __restrict__) (cond=0xffffa00003031d50, mutex=0xffffa00003031d28, ts=<optimized out>) at libc/pthread.cc:623
#9 0x0000100000c8d85e in os::PlatformEvent::park(long) [clone .part.0] ()
#10 0x0000100000c38529 in Monitor::IWait(Thread*, long) ()
#11 0x0000100000c38f41 in Monitor::wait(bool, long, bool) ()
#12 0x0000100000ea10f9 in WatcherThread::sleep() const ()
#13 0x0000100000ea11af in WatcherThread::run() ()
#14 0x0000100000ea75bf in Thread::call_run() ()
#15 0x0000100000c863b6 in thread_native_entry(Thread*) ()
#16 0x000000004045f6a6 in operator() (__closure=0xffffa00002ff8c00) at libc/pthread.cc:115
#17 std::__invoke_impl<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#18 std::__invoke_r<void, pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
#19 std::_Function_handler<void(), pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const pthread_private::thread_attr*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
#20 0x00000000403fbffa in sched::thread::main (this=0xffff800003774040) at core/sched.cc:1267
#21 sched::thread_main_c (t=0xffff800003774040) at arch/x64/arch-switch.hh:325
#22 0x0000000040390ec3 in thread_main () at arch/x64/entry.S:113
thread 0xffff800003972040 Common-Cleaner
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff800001855040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff800003972040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x2000196015c0, this=0x200019601500) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0xffffa00002f53700, user_mutex=0xffffa00003526880, tmr=tmr@entry=0x2000196015c0) at core/condvar.cc:43
#8 0x000000004046110d in pthread_cond_timedwait(pthread_cond_t * __restrict__, pthread_mutex_t * __restrict__, const timespec * __restrict__) (cond=0xffffa000034fb750, mutex=0xffffa000034fb728, ts=<optimized out>) at libc/pthread.cc:623
#9 0x0000100000c8d85e in os::PlatformEvent::park(long) [clone .part.0] ()
#10 0x0000100000c5d6d8 in ObjectMonitor::wait(long, bool, Thread*) ()
#11 0x0000100000e676e8 in ObjectSynchronizer::wait(Handle, long, Thread*) ()
#12 0x00001000009a182f in JVM_MonitorWait ()
#13 0x000020000920fe17 in ?? ()
#14 0x0000000000000000 in ?? ()
thread 0xffff800003e3a040 SeedGenerator T
#0 fill_ts<long, std::ratio<1, 1000000000> > (ts=0x2000197019a0, d=...) at /usr/include/c++/10/chrono:422
#1 clock_gettime (clk_id=<optimized out>, ts=0x2000197019a0) at libc/time.cc:81
#2 0x0000100000c7e580 in os::javaTimeNanos() ()
#3 0x0000200010cd0ce8 in ?? ()
#4 0x0000000800356780 in ?? ()
#5 0x0000000000000000 in ?? ()
thread 0xffff8000009f5040 SeedGenerator T
#0 sched::thread::switch_to (this=0x11225f0bd, this@entry=0xffff8000009ef040) at arch/x64/arch-switch.hh:108
#1 0x00000000403fb397 in sched::cpu::reschedule_from_interrupt (this=0xffff80000001e040, called_from_yield=<optimized out>, preempt_after=...) at core/sched.cc:374
#2 0x00000000403fb945 in sched::cpu::schedule () at include/osv/sched.hh:1358
#3 sched::thread::wait (this=this@entry=0xffff8000009f5040) at core/sched.cc:1273
#4 0x00000000403dbb01 in sched::thread::do_wait_until<sched::noninterruptible, sched::thread::dummy_lock, waiter::wait(sched::timer*) const::{lambda()#1}>(sched::thread::dummy_lock&, waiter::wait(sched::timer*) const::{lambda()#1}) (mtx=<synthetic pointer>..., pred=...) at include/osv/sched.hh:1114
#5 sched::thread::wait_until<waiter::wait(sched::timer*) const::{lambda()#1}>(waiter::wait(sched::timer*) const::{lambda()#1}) (pred=...) at include/osv/sched.hh:1125
#6 waiter::wait (tmr=0x200019901780, this=0x2000199016c0) at include/osv/wait_record.hh:46
#7 condvar::wait (this=this@entry=0xffffa00003fbe080, user_mutex=0xffffa00003eafb40, tmr=tmr@entry=0x200019901780) at core/condvar.cc:43
#8 0x000000004046110d in pthread_cond_timedwait(pthread_cond_t * __restrict__, pthread_mutex_t * __restrict__, const timespec * __restrict__) (cond=0xffffa00003ee5f50, mutex=0xffffa00003ee5f28, ts=<optimized out>) at libc/pthread.cc:623
#9 0x0000100000c8d85e in os::PlatformEvent::park(long) [clone .part.0] ()
#10 0x0000100000c8e966 in os::sleep(Thread*, long, bool) ()
#11 0x00001000009b61c7 in JVM_Sleep ()
#12 0x000020000920fe17 in ?? ()
#13 0x0000000800108530 in ?? ()
#14 0xffff9000041e0000 in ?? ()
#15 0x0000200019901990 in ?? ()
#16 0x0000000000000000 in ?? ()
The last two threads or actually the 2nd last (49 - SeedGenerator) is running and seems to be stuck in fill_ts<long, std::ratio<1, 1000000000> > (ts=0x2000197019a0, d=...) at /usr/include/c++/10/chrono:422. The fill_ts is an inline function implemented in libc/time.cc and I do not see how it can be stuck like this?
Is this some kind of a temporary deadlock situation?