Purgeable storage reporting crashed node_exporter with MacOS 15.3.2

[tickez]

          It appears that this feature creates an issue with latest MacOS 15.3.2. 

I get segmentation faults both in Intel and Apple Silicon Macs. Runs fine with 15.3
Disabling the filesystem collector resolves the crash.

SIGSEGV: segmentation violation
PC=0x19af21ff8 m=0 sigcode=2 addr=0x0
signal arrived during cgo execution

goroutine 63 gp=0x14000093c00 m=0 mp=0x100da2a80 [syscall]:
runtime.cgocall(0x1006c37c0, 0x1400017b418)
	runtime/cgocall.go:167 +0x44 fp=0x1400017b3e0 sp=0x1400017b3a0 pc=0x1001a0e34
github.com/prometheus/node_exporter/collector._Cfunc_purgeable(0x600001d54f00)
	_cgo_gotypes.go:731 +0x34 fp=0x1400017b410 sp=0x1400017b3e0 pc=0x1006708b4
github.com/prometheus/node_exporter/collector.(*filesystemCollector).GetStats(0x1400023bdc0)
	github.com/prometheus/node_exporter/collector/filesystem_macos.go:103 +0x338 fp=0x1400017b630 sp=0x1400017b410 pc=0x1006713a8
github.com/prometheus/node_exporter/collector.(*filesystemCollector).Update(0x1400023bdc0, 0x1400011f810)
	github.com/prometheus/node_exporter/collector/filesystem_common.go:186 +0x2c fp=0x1400017bd90 sp=0x1400017b630 pc=0x10066526c
github.com/prometheus/node_exporter/collector.execute({0x1006cbc7d, 0xa}, {0x100928dc0, 0x1400023bdc0}, 0x1400011f810, 0x1400011c430)
	github.com/prometheus/node_exporter/collector/collector.go:160 +0x60 fp=0x1400017bf50 sp=0x1400017bd90 pc=0x100663920
github.com/prometheus/node_exporter/collector.NodeCollector.Collect.func1({0x1006cbc7d?, 0x0?}, {0x100928dc0?, 0x1400023bdc0?})
	github.com/prometheus/node_exporter/collector/collector.go:151 +0x38 fp=0x1400017bfa0 sp=0x1400017bf50 pc=0x100663888
github.com/prometheus/node_exporter/collector.NodeCollector.Collect.gowrap1()
	github.com/prometheus/node_exporter/collector/collector.go:153 +0x3c fp=0x1400017bfd0 sp=0x1400017bfa0 pc=0x10066381c
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400017bfd0 sp=0x1400017bfd0 pc=0x1001abf54
created by github.com/prometheus/node_exporter/collector.NodeCollector.Collect in goroutine 56
	github.com/prometheus/node_exporter/collector/collector.go:150 +0xac

Originally posted by @tickez in #3206 (comment)

[ledeuns]

I can confirm v1.9.0 works on my machine (Mac mini M4) :

node_exporter, version 1.9.0 (branch: HEAD, revision: 02afa5c53c36123611533f2defea6ccd4546a9bb)
  build user:       root@48b969982efd
  build date:       20250217-07:25:44
  go version:       go1.23.6
  platform:         darwin/arm64
  tags:             unknown

ProductName:            macOS
ProductVersion:         15.3.2
BuildVersion:           24D81

Do you have some specific config ?

[tickez]

Nothing special, IMO. Installation through homebrew.

❯ brew info node_exporter
==> node_exporter: stable 1.9.0 (bottled), HEAD
Prometheus exporter for machine metrics
https://prometheus.io/
Installed
/opt/homebrew/Cellar/node_exporter/1.9.0 (12 files, 12.7MB) *
  Poured from bottle using the formulae.brew.sh API on 2025-02-18 at 21:47:41
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/n/node_exporter.rb

Go version is different...

node_exporter, version 1.9.0 (branch: , revision: unknown)
  build user:       Homebrew
  build date:       
  go version:       go1.24.0
  platform:         darwin/arm64
  tags:             unknown

[tickez]

Looks like the issue is with an smbfs mounted volume from my NAS.

--collector.filesystem.fs-types-include=apfs makes it work
--collector.filesystem.fs-types-include=smbfs makes it crash

[ledeuns]

There must be something else...

./node_exporter --web.listen-address=:9110 --collector.filesystem.fs-types-include=smbfs
time=2025-03-15T14:50:04.731Z level=INFO source=node_exporter.go:216 msg="Starting node_exporter" version="(version=1.9.0, branch=master, revision=0c10545e8dfb983f4c013281c4904be27f645de2)"
time=2025-03-15T14:50:04.731Z level=INFO source=node_exporter.go:217 msg="Build context" build_context="(go=go1.23.7, platform=darwin/arm64, user=user, date=20250315-14:43:46, tags=unknown)"
time=2025-03-15T14:50:04.732Z level=INFO source=filesystem_common.go:266 msg="Parsed flag --collector.filesystem.mount-points-exclude" collector=filesystem flag=^/(dev)($|/)
time=2025-03-15T14:50:04.732Z level=INFO source=filesystem_common.go:298 msg="Parsed flag --collector.filesystem.fs-types-include" collector=filesystem flag=smbfs
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:135 msg="Enabled collectors"
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=boottime
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=cpu
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=diskstats
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=filesystem
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=loadavg
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=meminfo
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=netdev
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=os
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=powersupplyclass
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=textfile
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=thermal
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=time
time=2025-03-15T14:50:04.732Z level=INFO source=node_exporter.go:141 msg=uname
time=2025-03-15T14:50:04.732Z level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9110
time=2025-03-15T14:50:04.732Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9110
time=2025-03-15T14:50:06.641Z level=ERROR source=collector.go:168 msg="collector failed" name=thermal duration_seconds=0.002650334 err="no CPU power status has been recorded"

curl localhost:9110/metrics | grep purge 
# HELP node_filesystem_purgeable_bytes Filesystem space available including purgeable space (MacOS specific).
# TYPE node_filesystem_purgeable_bytes gauge
node_filesystem_purgeable_bytes{device="//server/volume",device_error="",fstype="smbfs",mountpoint="/Volumes/volume"} 0
# HELP node_memory_purgeable_bytes Memory information field purgeable_bytes.
# TYPE node_memory_purgeable_bytes gauge
node_memory_purgeable_bytes 4.28949504e+08

[tickez]

The minute I enable the inclusion of my SMB mounted timemachine it crashes.

If I exclude that mount point it collects data from other SMB mounts.
--collector.filesystem.mount-points-exclude=.+/timemachine$

[logopk]

Nothing special, IMO. Installation through homebrew.

❯ brew info node_exporter
==> node_exporter: stable 1.9.0 (bottled), HEAD
Prometheus exporter for machine metrics
https://prometheus.io/
Installed
/opt/homebrew/Cellar/node_exporter/1.9.0 (12 files, 12.7MB) *
  Poured from bottle using the formulae.brew.sh API on 2025-02-18 at 21:47:41
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/n/node_exporter.rb

Go version is different...

node_exporter, version 1.9.0 (branch: , revision: unknown)
  build user:       Homebrew
  build date:       
  go version:       go1.24.0
  platform:         darwin/arm64
  tags:             unknown

Same setup is not running here in macOS 15.3.2 (24D81) on an M4 MacbookPro

Service is starting fine and is delivering metrics (incl. purgeable) for some time and then BANG, fails with segfault:

--web.config.file=/Users/.../nodeexporter-mac/node_exporter-mbp4.conf --collector.filesystem.fs-types-include=apfs

time=2025-03-18T12:51:50.259Z level=INFO source=node_exporter.go:141 msg=netdev
time=2025-03-18T12:51:50.259Z level=INFO source=node_exporter.go:141 msg=os
time=2025-03-18T12:51:50.259Z level=INFO source=node_exporter.go:141 msg=powersupplyclass
time=2025-03-18T12:51:50.259Z level=INFO source=node_exporter.go:141 msg=textfile
time=2025-03-18T12:51:50.259Z level=INFO source=node_exporter.go:141 msg=thermal
time=2025-03-18T12:51:50.259Z level=INFO source=node_exporter.go:141 msg=time
time=2025-03-18T12:51:50.259Z level=INFO source=node_exporter.go:141 msg=uname
time=2025-03-18T12:51:50.260Z level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9100
time=2025-03-18T12:51:50.262Z level=INFO source=tls_config.go:383 msg="TLS is enabled." http2=true address=[::]:9100
time=2025-03-18T12:51:54.653Z level=ERROR source=collector.go:168 msg="collector failed" name=thermal duration_seconds=0.000364417 err="no CPU power status has been recorded"



time=2025-03-18T12:52:09.599Z level=ERROR source=collector.go:168 msg="collector failed" name=thermal duration_seconds=0.001476084 err="no CPU power status has been recorded"
time=2025-03-18T12:52:24.604Z level=ERROR source=collector.go:168 msg="collector failed" name=thermal duration_seconds=0.000366042 err="no CPU power status has been recorded"
time=2025-03-18T12:52:39.610Z level=ERROR source=collector.go:168 msg="collector failed" name=thermal duration_seconds=0.001776959 err="no CPU power status has been recorded"

...


time=2025-03-19T08:31:39.619Z level=ERROR source=collector.go:168 msg="collector failed" name=thermal duration_seconds=0.002728625 err="no CPU power status has been recorded"
SIGSEGV: segmentation violation
PC=0x198d19ff8 m=5 sigcode=2 addr=0x0
signal arrived during cgo execution

goroutine 109804 gp=0x14000103180 m=5 mp=0x14000180008 [syscall]:
runtime.cgocall(0x1010af7c0, 0x14000205418)
	runtime/cgocall.go:167 +0x44 fp=0x140002053e0 sp=0x140002053a0 pc=0x100b8ce34
github.com/prometheus/node_exporter/collector._Cfunc_purgeable(0x60001766c780)
	_cgo_gotypes.go:731 +0x34 fp=0x14000205410 sp=0x140002053e0 pc=0x10105c8b4
github.com/prometheus/node_exporter/collector.(*filesystemCollector).GetStats(0x140000368c0)
	github.com/prometheus/node_exporter/collector/filesystem_macos.go:103 +0x338 fp=0x14000205630 sp=0x14000205410 pc=0x10105d3a8
github.com/prometheus/node_exporter/collector.(*filesystemCollector).Update(0x140000368c0, 0x14000368150)
	github.com/prometheus/node_exporter/collector/filesystem_common.go:186 +0x2c fp=0x14000205d90 sp=0x14000205630 pc=0x10105126c
github.com/prometheus/node_exporter/collector.execute({0x1010b7c7d, 0xa}, {0x101314dc0, 0x140000368c0}, 0x14000368150, 0x1400002a4b0)
	github.com/prometheus/node_exporter/collector/collector.go:160 +0x60 fp=0x14000205f50 sp=0x14000205d90 pc=0x10104f920
github.com/prometheus/node_exporter/collector.NodeCollector.Collect.func1({0x1010b7c7d?, 0x0?}, {0x101314dc0?, 0x140000368c0?})
	github.com/prometheus/node_exporter/collector/collector.go:151 +0x38 fp=0x14000205fa0 sp=0x14000205f50 pc=0x10104f888
github.com/prometheus/node_exporter/collector.NodeCollector.Collect.gowrap1()
	github.com/prometheus/node_exporter/collector/collector.go:153 +0x3c fp=0x14000205fd0 sp=0x14000205fa0 pc=0x10104f81c
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000205fd0 sp=0x14000205fd0 pc=0x100b97f54
created by github.com/prometheus/node_exporter/collector.NodeCollector.Collect in goroutine 109802
	github.com/prometheus/node_exporter/collector/collector.go:150 +0xac

goroutine 1 gp=0x140000021c0 m=nil [sync.WaitGroup.Wait, 1099 minutes]:
runtime.gopark(0x101796480?, 0x1018a00d0?, 0x0?, 0x80?, 0x18?)
	runtime/proc.go:435 +0xc8 fp=0x140002099a0 sp=0x14000209980 pc=0x100b904f8
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.semacquire1(0x14000097390, 0x0, 0x1, 0x0, 0x18)
	runtime/sema.go:188 +0x204 fp=0x140002099f0 sp=0x140002099a0 pc=0x100b6f6a4
sync.runtime_SemacquireWaitGroup(0x14000209a68?)
	runtime/sema.go:110 +0x2c fp=0x14000209a30 sp=0x140002099f0 pc=0x100b91b1c
sync.(*WaitGroup).Wait(0x14000097388)
	sync/waitgroup.go:118 +0x70 fp=0x14000209a50 sp=0x14000209a30 pc=0x100badf20
golang.org/x/sync/errgroup.(*Group).Wait(0x14000097380)
	golang.org/x/sync@v0.10.0/errgroup/errgroup.go:56 +0x28 fp=0x14000209a70 sp=0x14000209a50 pc=0x1010296c8
github.com/prometheus/exporter-toolkit/web.ServeMultiple({0x1400002b600, 0x1, 0x1010b2b74?}, 0x140000b4600, 0x1400000e0a8, 0x1400002a4b0)
	github.com/prometheus/exporter-toolkit@v0.14.0/web/tls_config.go:277 +0x104 fp=0x14000209ac0 sp=0x14000209a70 pc=0x10102c944
github.com/prometheus/exporter-toolkit/web.ListenAndServe(0x140000b4600, 0x1400000e0a8, 0x1400002a4b0)
	github.com/prometheus/exporter-toolkit@v0.14.0/web/tls_config.go:325 +0x3bc fp=0x14000209b60 sp=0x14000209ac0 pc=0x10102cd8c
main.main()
	github.com/prometheus/node_exporter/node_exporter.go:246 +0x1098 fp=0x14000209f40 sp=0x14000209b60 pc=0x101062908
runtime.main()
	runtime/proc.go:283 +0x284 fp=0x14000209fd0 sp=0x14000209f40 pc=0x100b5adf4
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000209fd0 sp=0x14000209fd0 pc=0x100b97f54

goroutine 2 gp=0x14000002c40 m=nil [force gc (idle), 114 minutes]:
runtime.gopark(0x1aa7e4ec46e16?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xc8 fp=0x14000068f90 sp=0x14000068f70 pc=0x100b904f8
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.forcegchelper()
	runtime/proc.go:348 +0xb8 fp=0x14000068fd0 sp=0x14000068f90 pc=0x100b5b148
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000068fd0 sp=0x14000068fd0 pc=0x100b97f54
created by runtime.init.7 in goroutine 1
	runtime/proc.go:336 +0x24

goroutine 3 gp=0x14000003180 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xc8 fp=0x14000069760 sp=0x14000069740 pc=0x100b904f8
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.bgsweep(0x14000094000)
	runtime/mgcsweep.go:316 +0x108 fp=0x140000697b0 sp=0x14000069760 pc=0x100b43b18
runtime.gcenable.gowrap1()
	runtime/mgc.go:204 +0x28 fp=0x140000697d0 sp=0x140000697b0 pc=0x100b37928
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140000697d0 sp=0x140000697d0 pc=0x100b97f54
created by runtime.gcenable in goroutine 1
	runtime/mgc.go:204 +0x6c

goroutine 4 gp=0x14000003340 m=nil [GC scavenge wait]:
runtime.gopark(0xabf77?, 0x8432d?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xc8 fp=0x14000069f60 sp=0x14000069f40 pc=0x100b904f8
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.(*scavengerState).park(0x10178de40)
	runtime/mgcscavenge.go:425 +0x5c fp=0x14000069f90 sp=0x14000069f60 pc=0x100b415bc
runtime.bgscavenge(0x14000094000)
	runtime/mgcscavenge.go:658 +0xac fp=0x14000069fb0 sp=0x14000069f90 pc=0x100b41b5c
runtime.gcenable.gowrap2()
	runtime/mgc.go:205 +0x28 fp=0x14000069fd0 sp=0x14000069fb0 pc=0x100b378c8
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000069fd0 sp=0x14000069fd0 pc=0x100b97f54
created by runtime.gcenable in goroutine 1
	runtime/mgc.go:205 +0xac

goroutine 18 gp=0x14000182700 m=nil [finalizer wait, 1099 minutes]:
runtime.gopark(0x18000685c8?, 0x101ba8328?, 0xd8?, 0x85?, 0x100ba0d34?)
	runtime/proc.go:435 +0xc8 fp=0x14000068590 sp=0x14000068570 pc=0x100b904f8
runtime.runfinq()
	runtime/mfinal.go:196 +0x108 fp=0x140000687d0 sp=0x14000068590 pc=0x100b36928
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140000687d0 sp=0x140000687d0 pc=0x100b97f54
created by runtime.createfing in goroutine 1
	runtime/mfinal.go:166 +0x80

goroutine 19 gp=0x14000182e00 m=nil [chan receive, 1 minutes]:
runtime.gopark(0x140001aedc0?, 0x14000452630?, 0x48?, 0x47?, 0x100ca3fd8?)
	runtime/proc.go:435 +0xc8 fp=0x140000646f0 sp=0x140000646d0 pc=0x100b904f8
runtime.chanrecv(0x140001902a0, 0x0, 0x1)
	runtime/chan.go:664 +0x42c fp=0x14000064770 sp=0x140000646f0 pc=0x100b2901c
runtime.chanrecv1(0x0?, 0x0?)
	runtime/chan.go:506 +0x14 fp=0x140000647a0 sp=0x14000064770 pc=0x100b28bb4
runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
	runtime/mgc.go:1796
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
	runtime/mgc.go:1799 +0x3c fp=0x140000647d0 sp=0x140000647a0 pc=0x100b3ab4c
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140000647d0 sp=0x140000647d0 pc=0x100b97f54
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
	runtime/mgc.go:1794 +0x78

goroutine 34 gp=0x14000183180 m=nil [IO wait, 112 minutes]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xc8 fp=0x140002af7f0 sp=0x140002af7d0 pc=0x100b904f8
runtime.netpollblock(0x140002af888?, 0xc06300?, 0x1?)
	runtime/netpoll.go:575 +0x158 fp=0x140002af830 sp=0x140002af7f0 pc=0x100b54458
internal/poll.runtime_pollWait(0x1285dc3a0, 0x72)
	runtime/netpoll.go:351 +0xa0 fp=0x140002af860 sp=0x140002af830 pc=0x100b8f6b0
internal/poll.(*pollDesc).wait(0x140000b2700?, 0x100c08558?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x140002af890 sp=0x140002af860 pc=0x100c01b18
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0x140000b2700)
	internal/poll/fd_unix.go:620 +0x24c fp=0x140002af940 sp=0x140002af890 pc=0x100c063ec
net.(*netFD).accept(0x140000b2700)
	net/fd_unix.go:172 +0x28 fp=0x140002afa00 sp=0x140002af940 pc=0x100ccfcb8
net.(*TCPListener).accept(0x14000097340)
	net/tcpsock_posix.go:159 +0x24 fp=0x140002afa50 sp=0x140002afa00 pc=0x100ce3df4
net.(*TCPListener).Accept(0x14000097340)
	net/tcpsock.go:380 +0x2c fp=0x140002afa90 sp=0x140002afa50 pc=0x100ce30dc
crypto/tls.(*listener).Accept(0x1400000ef30)
	crypto/tls/tls.go:67 +0x2c fp=0x140002afae0 sp=0x140002afa90 pc=0x100d6790c
net/http.(*onceCloseListener).Accept(0x140000c0000?)
	<autogenerated>:1 +0x30 fp=0x140002afb00 sp=0x140002afae0 pc=0x100e10c40
net/http.(*Server).Serve(0x140000b4600, {0x101318f60, 0x1400000ef30})
	net/http/server.go:3424 +0x290 fp=0x140002afc30 sp=0x140002afb00 pc=0x100de96a0
net/http.(*Server).ServeTLS(0x140000b4600, {0x1013190b0, 0x14000097340}, {0x0, 0x0}, {0x0, 0x0})
	net/http/server.go:3493 +0x2cc fp=0x140002afd90 sp=0x140002afc30 pc=0x100de9e1c
github.com/prometheus/exporter-toolkit/web.Serve({0x1013190b0, 0x14000097340}, 0x140000b4600, 0x1400000e0a8, 0x1400002a4b0)
	github.com/prometheus/exporter-toolkit@v0.14.0/web/tls_config.go:405 +0x4f0 fp=0x140002aff20 sp=0x140002afd90 pc=0x10102d3e0
github.com/prometheus/exporter-toolkit/web.ServeMultiple.func1()
	github.com/prometheus/exporter-toolkit@v0.14.0/web/tls_config.go:274 +0x30 fp=0x140002aff60 sp=0x140002aff20 pc=0x10102c9b0
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78 +0x54 fp=0x140002affd0 sp=0x140002aff60 pc=0x101029824
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140002affd0 sp=0x140002affd0 pc=0x100b97f54
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
	golang.org/x/sync@v0.10.0/errgroup/errgroup.go:75 +0x94

goroutine 98616 gp=0x140000036c0 m=nil [IO wait]:
runtime.gopark(0xffffffffffffffff?, 0xffffffffffffffff?, 0x23?, 0x0?, 0x100be5da0?)
	runtime/proc.go:435 +0xc8 fp=0x140002937d0 sp=0x140002937b0 pc=0x100b904f8
runtime.netpollblock(0x0?, 0x0?, 0x0?)
	runtime/netpoll.go:575 +0x158 fp=0x14000293810 sp=0x140002937d0 pc=0x100b54458
internal/poll.runtime_pollWait(0x1285dc170, 0x72)
	runtime/netpoll.go:351 +0xa0 fp=0x14000293840 sp=0x14000293810 pc=0x100b8f6b0
internal/poll.(*pollDesc).wait(0x140000b2300?, 0x140000f4880?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x14000293870 sp=0x14000293840 pc=0x100c01b18
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140000b2300, {0x140000f4880, 0xd80, 0xd80})
	internal/poll/fd_unix.go:165 +0x1fc fp=0x14000293910 sp=0x14000293870 pc=0x100c02dcc
net.(*netFD).Read(0x140000b2300, {0x140000f4880?, 0x14000293998?, 0x100b300fc?})
	net/fd_posix.go:55 +0x28 fp=0x14000293960 sp=0x14000293910 pc=0x100cce288
net.(*conn).Read(0x140001aa060, {0x140000f4880?, 0x301894108?, 0x1018a3410?})
	net/net.go:194 +0x34 fp=0x140002939b0 sp=0x14000293960 pc=0x100cdb334
crypto/tls.(*atLeastReader).Read(0x14000333b48, {0x140000f4880?, 0x14000293a58?, 0x100d29674?})
	crypto/tls/conn.go:809 +0x40 fp=0x14000293a00 sp=0x140002939b0 pc=0x100d29510
bytes.(*Buffer).ReadFrom(0x140000a6638, {0x101315140, 0x14000333b48})
	bytes/buffer.go:211 +0x90 fp=0x14000293a60 sp=0x14000293a00 pc=0x100c26120
crypto/tls.(*Conn).readFromUntil(0x140000a6388, {0x1285e0e18, 0x140001aa060}, 0x14000293b00?)
	crypto/tls/conn.go:831 +0xcc fp=0x14000293aa0 sp=0x14000293a60 pc=0x100d296dc
crypto/tls.(*Conn).readRecordOrCCS(0x140000a6388, 0x0)
	crypto/tls/conn.go:629 +0x35c fp=0x14000293d20 sp=0x14000293aa0 pc=0x100d26e8c
crypto/tls.(*Conn).readRecord(...)
	crypto/tls/conn.go:591
crypto/tls.(*Conn).Read(0x140000a6388, {0x140000e4ac0, 0x9, 0x100db0ac0?})
	crypto/tls/conn.go:1385 +0x15c fp=0x14000293d90 sp=0x14000293d20 pc=0x100d2cd5c
io.ReadAtLeast({0x1285ed738, 0x140000a6388}, {0x140000e4ac0, 0x9, 0x9}, 0x9)
	io/io.go:335 +0xa0 fp=0x14000293de0 sp=0x14000293d90 pc=0x100bdaa70
io.ReadFull(...)
	io/io.go:354
net/http.http2readFrameHeader({0x140000e4ac0, 0x9, 0x14000458f30?}, {0x1285ed738?, 0x140000a6388?})
	net/http/h2_bundle.go:1805 +0x58 fp=0x14000293e30 sp=0x14000293de0 pc=0x100da5258
net/http.(*http2Framer).ReadFrame(0x140000e4a80)
	net/http/h2_bundle.go:2072 +0x74 fp=0x14000293ee0 sp=0x14000293e30 pc=0x100da5924
net/http.(*http2serverConn).readFrames(0x14000200b40)
	net/http/h2_bundle.go:4906 +0x98 fp=0x14000293fb0 sp=0x14000293ee0 pc=0x100db0a28
net/http.(*http2serverConn).serve.gowrap8()
	net/http/h2_bundle.go:5029 +0x28 fp=0x14000293fd0 sp=0x14000293fb0 pc=0x100db1f78
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000293fd0 sp=0x14000293fd0 pc=0x100b97f54
created by net/http.(*http2serverConn).serve in goroutine 98613
	net/http/h2_bundle.go:5029 +0x524

goroutine 98613 gp=0x14000003dc0 m=nil [select]:
runtime.gopark(0x1400002f700?, 0x5?, 0x8?, 0xf2?, 0x1400002f32e?)
	runtime/proc.go:435 +0xc8 fp=0x1400002f1a0 sp=0x1400002f180 pc=0x100b904f8
runtime.selectgo(0x1400002f700, 0x1400002f324, 0x14000458f60?, 0x0, 0x0?, 0x1)
	runtime/select.go:351 +0x6c4 fp=0x1400002f2d0 sp=0x1400002f1a0 pc=0x100b6e834
net/http.(*http2serverConn).serve(0x14000200b40, {0xfa, 0x1000, 0x1000, 0x100000, 0x100000, 0x100000, 0x0, 0x37e11d600, 0x0, ...})
	net/http/h2_bundle.go:5038 +0x7e4 fp=0x1400002f7a0 sp=0x1400002f2d0 pc=0x100db17f4
net/http.(*http2Server).serveConn(0x140002635e0, {0x10131deb0, 0x140000a6388}, 0x1400002fae8, 0x0)
	net/http/h2_bundle.go:4614 +0x9b8 fp=0x1400002fab0 sp=0x1400002f7a0 pc=0x100dafc68
net/http.(*http2Server).ServeConn(...)
	net/http/h2_bundle.go:4473
net/http.http2ConfigureServer.func1(0x140000b4600, {0x10131deb0, 0x140000a6388}, {0x1013145d8, 0x14000112bc0}, 0x0)
	net/http/h2_bundle.go:4373 +0x110 fp=0x1400002fb40 sp=0x1400002fab0 pc=0x100daf1c0
net/http.http2ConfigureServer.func2(0x1012b4620?, 0x1400002fe98?, {0x1013145d8?, 0x14000112bc0?})
	net/http/h2_bundle.go:4381 +0x44 fp=0x1400002fb80 sp=0x1400002fb40 pc=0x100daf084
net/http.(*conn).serve(0x140000c0000, {0x1013199b0, 0x140000f8960})
	net/http/server.go:2000 +0xfc8 fp=0x1400002ffa0 sp=0x1400002fb80 pc=0x100de5238
net/http.(*Server).Serve.gowrap3()
	net/http/server.go:3454 +0x30 fp=0x1400002ffd0 sp=0x1400002ffa0 pc=0x100de9a30
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400002ffd0 sp=0x1400002ffd0 pc=0x100b97f54
created by net/http.(*Server).Serve in goroutine 34
	net/http/server.go:3454 +0x3d8

goroutine 85 gp=0x14000102c40 m=nil [GC worker (idle)]:
runtime.gopark(0x1b09b249f2405?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xc8 fp=0x140002a7f10 sp=0x140002a7ef0 pc=0x100b904f8
runtime.gcBgMarkWorker(0x14000190bd0)
	runtime/mgc.go:1423 +0xdc fp=0x140002a7fb0 sp=0x140002a7f10 pc=0x100b39dbc
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x28 fp=0x140002a7fd0 sp=0x140002a7fb0 pc=0x100b39ca8
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140002a7fd0 sp=0x140002a7fd0 pc=0x100b97f54
created by runtime.gcBgMarkStartWorkers in goroutine 73
	runtime/mgc.go:1339 +0x140

goroutine 109802 gp=0x14000103880 m=nil [sync.WaitGroup.Wait]:
runtime.gopark(0x10179c180?, 0x100b27d18?, 0xa0?, 0x7d?, 0x1286017b8?)
	runtime/proc.go:435 +0xc8 fp=0x14000078d70 sp=0x14000078d50 pc=0x100b904f8
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.semacquire1(0x140004850e8, 0x0, 0x1, 0x0, 0x18)
	runtime/sema.go:188 +0x204 fp=0x14000078dc0 sp=0x14000078d70 pc=0x100b6f6a4
sync.runtime_SemacquireWaitGroup(0x14000078e80?)
	runtime/sema.go:110 +0x2c fp=0x14000078e00 sp=0x14000078dc0 pc=0x100b91b1c
sync.(*WaitGroup).Wait(0x140004850e0)
	sync/waitgroup.go:118 +0x70 fp=0x14000078e20 sp=0x14000078e00 pc=0x100badf20
github.com/prometheus/node_exporter/collector.NodeCollector.Collect({0x14000020e70?, 0x1400002a4b0?}, 0x14000368150)
	github.com/prometheus/node_exporter/collector/collector.go:155 +0x1ac fp=0x14000078ef0 sp=0x14000078e20 pc=0x10104f7ac
github.com/prometheus/node_exporter/collector.(*NodeCollector).Collect(0x14000078f80?, 0x14000078f58?)
	<autogenerated>:1 +0x3c fp=0x14000078f20 sp=0x14000078ef0 pc=0x10106059c
github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
	github.com/prometheus/client_golang@v1.20.5/prometheus/registry.go:456 +0xd8 fp=0x14000078fd0 sp=0x14000078f20 pc=0x100fa8cf8
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000078fd0 sp=0x14000078fd0 pc=0x100b97f54
created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather in goroutine 109796
	github.com/prometheus/client_golang@v1.20.5/prometheus/registry.go:548 +0x978

goroutine 109796 gp=0x14000182c40 m=nil [select]:
runtime.gopark(0x1400020b000?, 0x2?, 0x40?, 0xb4?, 0x1400020ac14?)
	runtime/proc.go:435 +0xc8 fp=0x1400020aa80 sp=0x1400020aa60 pc=0x100b904f8
runtime.selectgo(0x1400020b000, 0x1400020ac10, 0x1400020aed0?, 0x0, 0x0?, 0x1)
	runtime/select.go:351 +0x6c4 fp=0x1400020abb0 sp=0x1400020aa80 pc=0x100b6e834
github.com/prometheus/client_golang/prometheus.(*Registry).Gather(0x1400009a6e0)
	github.com/prometheus/client_golang@v1.20.5/prometheus/registry.go:523 +0x7f0 fp=0x1400020b0a0 sp=0x1400020abb0 pc=0x100fa8860
github.com/prometheus/client_golang/prometheus.Gatherers.Gather({0x14000089100, 0x2, 0x1400020b498?})
	github.com/prometheus/client_golang@v1.20.5/prometheus/registry.go:754 +0x11c fp=0x1400020b450 sp=0x1400020b0a0 pc=0x100fa9b7c
github.com/prometheus/client_golang/prometheus.(*Gatherers).Gather(0x1400020b500?)
	<autogenerated>:1 +0x38 fp=0x1400020b480 sp=0x1400020b450 pc=0x100fb12f8
github.com/prometheus/client_golang/prometheus.(*noTransactionGatherer).Gather(0x140004c73e0?)
	github.com/prometheus/client_golang@v1.20.5/prometheus/registry.go:1074 +0x28 fp=0x1400020b4a0 sp=0x1400020b480 pc=0x100faba88
github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1({0x1285f39f0, 0x14000125c80}, 0x14000226500)
	github.com/prometheus/client_golang@v1.20.5/prometheus/promhttp/http.go:165 +0x260 fp=0x1400020b6c0 sp=0x1400020b4a0 pc=0x100fd9d90
net/http.HandlerFunc.ServeHTTP(0x140001b2f40?, {0x1285f39f0?, 0x14000125c80?}, 0x14000459170?)
	net/http/server.go:2294 +0x38 fp=0x1400020b6f0 sp=0x1400020b6c0 pc=0x100de5ff8
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentMetricHandler.InstrumentHandlerInFlight.func1({0x1285f39f0, 0x14000125c80}, 0x14000226500)
	github.com/prometheus/client_golang@v1.20.5/prometheus/promhttp/instrument_server.go:60 +0xa4 fp=0x1400020b770 sp=0x1400020b6f0 pc=0x100fdad34
net/http.HandlerFunc.ServeHTTP(0x101318e70?, {0x1285f39f0?, 0x14000125c80?}, 0x140004c8518?)
	net/http/server.go:2294 +0x38 fp=0x1400020b7a0 sp=0x1400020b770 pc=0x100de5ff8
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1({0x101318e70?, 0x1400006c6a8?}, 0x14000226500)
	github.com/prometheus/client_golang@v1.20.5/prometheus/promhttp/instrument_server.go:147 +0xa8 fp=0x1400020b8b0 sp=0x1400020b7a0 pc=0x100fdba38
net/http.HandlerFunc.ServeHTTP(0x1400002a4b0?, {0x101318e70?, 0x1400006c6a8?}, 0xfffffffffffffffc?)
	net/http/server.go:2294 +0x38 fp=0x1400020b8e0 sp=0x1400020b8b0 pc=0x100de5ff8
main.(*handler).ServeHTTP(0x1400009a4b0, {0x101318e70, 0x1400006c6a8}, 0x14000226500)
	github.com/prometheus/node_exporter/node_exporter.go:87 +0x204 fp=0x1400020ba30 sp=0x1400020b8e0 pc=0x101060d94
net/http.(*ServeMux).ServeHTTP(0x16f2ebccd?, {0x101318e70, 0x1400006c6a8}, 0x14000226500)
	net/http/server.go:2822 +0x1b4 fp=0x1400020ba80 sp=0x1400020ba30 pc=0x100de7b84
github.com/prometheus/exporter-toolkit/web.(*webHandler).ServeHTTP(0x140000efa40, {0x101318e70, 0x1400006c6a8}, 0x14000226500)
	github.com/prometheus/exporter-toolkit@v0.14.0/web/handler.go:102 +0x2f8 fp=0x1400020bcb0 sp=0x1400020ba80 pc=0x10102a888
net/http.serverHandler.ServeHTTP({0x100b98e10?}, {0x101318e70?, 0x1400006c6a8?}, 0x14000100008?)
	net/http/server.go:3301 +0xbc fp=0x1400020bce0 sp=0x1400020bcb0 pc=0x100e0492c
net/http.initALPNRequest.ServeHTTP({{0x1013199b0?, 0x140000201e0?}, 0x140000a6388?, {0x140000b4600?}}, {0x101318e70, 0x1400006c6a8}, 0x14000226500)
	net/http/server.go:3968 +0x1b4 fp=0x1400020be90 sp=0x1400020bce0 pc=0x100dec2b4
net/http.(*initALPNRequest).ServeHTTP(0x14000078f08?, {0x101318e70?, 0x1400006c6a8?}, 0x1400002a4b0?)
	<autogenerated>:1 +0x5c fp=0x1400020bee0 sp=0x1400020be90 pc=0x100e10eec
net/http.Handler.ServeHTTP-fm({0x101318e70?, 0x1400006c6a8?}, 0x14000078f80?)
	<autogenerated>:1 +0x48 fp=0x1400020bf10 sp=0x1400020bee0 pc=0x100e120a8
net/http.(*http2serverConn).runHandler(0x1010b1bcd?, 0x0?, 0x101314d40?, 0x140000b2280?)
	net/http/h2_bundle.go:6529 +0xe8 fp=0x1400020bfa0 sp=0x1400020bf10 pc=0x100db8c98
net/http.(*http2serverConn).scheduleHandler.gowrap1()
	net/http/h2_bundle.go:6463 +0x34 fp=0x1400020bfd0 sp=0x1400020bfa0 pc=0x100db8884
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400020bfd0 sp=0x1400020bfd0 pc=0x100b97f54
created by net/http.(*http2serverConn).scheduleHandler in goroutine 98613
	net/http/h2_bundle.go:6463 +0x208

goroutine 109801 gp=0x14000183880 m=nil [sync.WaitGroup.Wait]:
runtime.gopark(0x10179c080?, 0x14000037180?, 0x40?, 0x7a?, 0x100db349c?)
	runtime/proc.go:435 +0xc8 fp=0x1400007def0 sp=0x1400007ded0 pc=0x100b904f8
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.semacquire1(0x140004850d8, 0x0, 0x1, 0x0, 0x18)
	runtime/sema.go:188 +0x204 fp=0x1400007df40 sp=0x1400007def0 pc=0x100b6f6a4
sync.runtime_SemacquireWaitGroup(0x1400007dfa0?)
	runtime/sema.go:110 +0x2c fp=0x1400007df80 sp=0x1400007df40 pc=0x100b91b1c
sync.(*WaitGroup).Wait(0x140004850d0)
	sync/waitgroup.go:118 +0x70 fp=0x1400007dfa0 sp=0x1400007df80 pc=0x100badf20
github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func2()
	github.com/prometheus/client_golang@v1.20.5/prometheus/registry.go:473 +0x30 fp=0x1400007dfd0 sp=0x1400007dfa0 pc=0x100fa8bf0
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400007dfd0 sp=0x1400007dfd0 pc=0x100b97f54
created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather in goroutine 109796
	github.com/prometheus/client_golang@v1.20.5/prometheus/registry.go:472 +0x534

r0      0x12
r1      0x0
r2      0x1503
r3      0x1503
r4      0xa
r5      0xed49f019
r6      0x480
r7      0x20cc4d8e5
r8      0x3b71fd8f2fc5002f
r9      0x60000fb00483
r10     0x1000002024e8719
r11     0xa0080001
r12     0xf00ffffffffffff
r13     0x600016de3960
r14     0x600016de3960
r15     0x18
r16     0x198bd2270
r17     0x198797130
r18     0x0
r19     0x20379b978
r20     0x0
r21     0x14000216238
r22     0x140002ec0a0
r23     0x140002ec088
r24     0x1012927c0
r25     0x0
r26     0x0
r27     0x820
r28     0x140001821c0
r29     0x17130ee30
lr      0xa32a000198bd22a8
sp      0x17130ee30
pc      0x198d19ff8
fault   0x0
^C

▶ curl https://textmachine:9100/metrics --cacert /Users/xxx/chain.pem --cert /Users/xxx/client.crt --cert-type PEM --key /Users/xxx/client.key  | grep purge 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 51436    0 51436    0     0  1172k      0 --:--:-- --:--:-- --:--:-- 1195k
# HELP node_filesystem_purgeable_bytes Filesystem space available including purgeable space (MacOS specific).
# TYPE node_filesystem_purgeable_bytes gauge
node_filesystem_purgeable_bytes{device="/dev/disk1s1",device_error="",fstype="apfs",mountpoint="/System/Volumes/iSCPreboot"} 0
node_filesystem_purgeable_bytes{device="/dev/disk1s2",device_error="",fstype="apfs",mountpoint="/System/Volumes/xarts"} 0
node_filesystem_purgeable_bytes{device="/dev/disk1s3",device_error="",fstype="apfs",mountpoint="/System/Volumes/Hardware"} 0
node_filesystem_purgeable_bytes{device="/dev/disk3s1s1",device_error="",fstype="apfs",mountpoint="/"} 1.13448808991e+11
node_filesystem_purgeable_bytes{device="/dev/disk3s2",device_error="",fstype="apfs",mountpoint="/System/Volumes/Preboot"} 0
node_filesystem_purgeable_bytes{device="/dev/disk3s3",device_error="",fstype="apfs",mountpoint="/Volumes/Recovery"} 0
node_filesystem_purgeable_bytes{device="/dev/disk3s4",device_error="",fstype="apfs",mountpoint="/System/Volumes/Update"} 0
node_filesystem_purgeable_bytes{device="/dev/disk3s5",device_error="",fstype="apfs",mountpoint="/System/Volumes/Data"} 1.13448808991e+11
node_filesystem_purgeable_bytes{device="/dev/disk3s6",device_error="",fstype="apfs",mountpoint="/System/Volumes/VM"} 0
# HELP node_memory_purgeable_bytes Memory information field purgeable_bytes.
# TYPE node_memory_purgeable_bytes gauge
node_memory_purgeable_bytes 2.354020352e+09

[tickez]

I can confirm that even with the timemachine mount point excluded, the process eventually crashes after some time. In my case it ran ok for a couple days before diving.

time=2025-03-15T23:01:15.331Z level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9100
time=2025-03-15T23:01:15.331Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9100
time=2025-03-15T23:01:23.671Z level=ERROR source=collector.go:168 msg="collector failed" name=thermal duration_seconds=0.003384833 err="no CPU power status has been recorded"

...

time=2025-03-20T07:21:14.202Z level=ERROR source=collector.go:168 msg="collector failed" name=thermal duration_seconds=0.000784667 err="no CPU power status has been recorded"
SIGSEGV: segmentation violation
PC=0x1957a5ff8 m=3 sigcode=2 addr=0x0
signal arrived during cgo execution

[SuperQ]

I've merged the fix to main, let us know if it helps.

[tickez]

I've had my branch running in production for multiple days stable. Now deployed main and so far no issues.