Skip to content

Conversation

@giordano
Copy link
Member

@giordano giordano commented Nov 21, 2025

Reverts #60105. Nightly builds of aarch64-darwin Julia hang at startup on some systems, notably on GitHub Actions, making all nightly jobs timeout (after 6 hours...), see https://discourse.julialang.org/t/ci-testing-hangs-on-macos-nightly/133909 (previously reported on Slack at https://julialang.slack.com/archives/CPWJ5DGG1/p1763055610279379). See also julia-actions/julia-runtest#155. The error message when the process receives a SIGTERM signal is

in expression starting at none:0
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0) at (unknown file)
__psynch_mutexwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0) at (unknown file)
Allocations: 1 (Pool: 1; Big: 0); GC: 0

I can reliably reproduce it locally on a M1 box with MacOSX 12.6 21.6.0, I can provide more information as needed.

I bisected the issue to #60105. CC @xal-0 for your information.

@giordano giordano added the revert This reverts a previously merged PR. label Nov 21, 2025
@giordano
Copy link
Member Author

On the machine where I can reproduce the hang using builds from buildkite, I also systematically can't compile latest master (9af9b15) because I get the error

    JULIA usr/lib/julia/basecompiler-o.a
ERROR: rethrow() not allowed outside a catch block
make[1]: *** [/Volumes/STORAGE/mose/repo/julia/usr/lib/julia/basecompiler-o.a] Error 1
make: *** [julia-sysimg-release] Error 2

which @xal-0 mentioned on Slack is happening also on CI here quite frequently when building Julia. This revert PR also resolves that issue for me. Looks like #60105 had quite a few issues on aarch64-darwin

@giordano
Copy link
Member Author

giordano commented Nov 21, 2025

And just to further confirm this PR resolves the hang:

$ curl -LsfO https://buildkite.com/organizations/julialang/pipelines/julia-master/builds/52235/jobs/019aa86e-9eba-44e6-8a8f-09e9120a3250/artifacts/019aa87a-47dc-4ebc-b32a-d8a74145df20
$ tar xzf 019aa87a-47dc-4ebc-b32a-d8a74145df20 
$ julia-305ae33084/bin/julia 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.14.0-DEV.1278 (2025-11-21)
 _/ |\__'_|_|_|\__'_|  |  revert-60105-jitlink-cgmemmgr/305ae33084f (fork: 1 commits, 0 days)
|__/                   |

julia> 

Also, my local build, now successful:

$ ./julia 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.14.0-DEV.1278 (2025-11-21)
 _/ |\__'_|_|_|\__'_|  |  revert-60105-jitlink-cgmemmgr/305ae33084 (fork: 1 commits, 0 days)
|__/                   |

julia> 

@giordano giordano merged commit ddf1b02 into master Nov 22, 2025
11 checks passed
@giordano giordano deleted the revert-60105-jitlink-cgmemmgr branch November 22, 2025 01:20
@giordano
Copy link
Member Author

giordano commented Nov 22, 2025

For the record, prompted by a request from @xal-0 I found that System Integrity Protection is disabled on both systems where I could reproduce the hang (GitHub Actions and another remote M1 box I have access to):

$ csrutil status
System Integrity Protection status: disabled.

while it's enabled on other aarch64-darwin machines where I can't reproduced the issue (this is also the default, so most users will probably not be affected):

% csrutil status 
System Integrity Protection status: enabled.

The correlation between SIP being disabled and the issues is quite strong for the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

revert This reverts a previously merged PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants