Skip to content

VirtualThreadMetrics misreports live virtual threads count #6504

@alekkol

Description

@alekkol

Describe the bug

VirtualThreadMetrics adds new metric jvm.threads.virtual.live with 2 tags: mounted and queued. Micrometer documentation states

Note that aggregating the values of jvm.threads.virtual.live across the different tags gives the total number of virtual threads started but not ended.

The metric’s description is misleading: parked virtual threads are counted as neither mounted nor queued. A virtual thread that has started but is not currently running on a carrier thread is therefore excluded. As a result, this metric cannot answer the real question: how many virtual threads are alive in the JVM right now? That number matters, because every virtual thread that has started but not yet finished represents a potential memory leak.

Environment

  • Micrometer version: micrometer-java21-1.15.1
  • OS: Linux 6.1.132-147.221.amzn2023.x86_64
  • Java version: OpenJDK Runtime Environment Temurin-24.0.1+9 (build 24.0.1+9)

To Reproduce
How to reproduce the bug:

  1. Run the following code using JDK 24.0.1
public static void main(String[] args) {
    var queue = new SynchronousQueue<>();
    for (int i = 0; i < 10; i++) {
        Thread.ofVirtual().name("find-me").start(() -> {
            try {
                queue.take();
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
        });
    }

    new VirtualThreadMetrics().bindTo(new LoggingMeterRegistry());
}
  1. Create thread dump to get all threads (both platform and virtual)
jcmd $PID Thread.dump_to_file /tmp/dump.txt
  1. Find all virtual threads in the dump
cat /tmp/dump.txt | grep virtual
#34 "find-me" virtual
#36 "find-me" virtual
#37 "find-me" virtual
#39 "find-me" virtual
#41 "find-me" virtual
#26 "find-me" virtual
#43 "find-me" virtual
#28 "find-me" virtual
#44 "find-me" virtual
#31 "find-me" virtual
  1. But the metrics doesn't count these threads
INFO io.micrometer.core.instrument.logging.LoggingMeterRegistry -- jvm.threads.virtual.live{scheduling.status=mounted} value=0 threads
INFO io.micrometer.core.instrument.logging.LoggingMeterRegistry -- jvm.threads.virtual.live{scheduling.status=queued} value=0 threads
INFO io.micrometer.core.instrument.logging.LoggingMeterRegistry -- jvm.threads.virtual.parallelism{} value=2
INFO io.micrometer.core.instrument.logging.LoggingMeterRegistry -- jvm.threads.virtual.pool.size{} value=0 threads

Expected behavior
jvm.threads.virtual.live should accurately report the number of virtual threads that have started but not yet completed, exactly as the documentation specifies.

Additional context
The earlier implementation of VirtualThreadMetrics (see PR #6009) did not exhibit this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    instrumentationAn issue that is related to instrumenting a component

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions