Could not open library 'sodium': dlopen(sodium, 5): image not found. (LoadError) on Apple Silicon M1

[OleMchls]

Hey there,

I'm trying to get rbnacl to function on my M1, but I am hitting the following error message:

ole@Oles-MBP-2 test % ruby -e "require 'rbnacl'" 
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:10: warning: already initialized constant RbNaCl::Sodium::Version::MINIMUM_LIBSODIUM_VERSION
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:10: warning: previous definition of MINIMUM_LIBSODIUM_VERSION was here
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:11: warning: already initialized constant RbNaCl::Sodium::Version::MINIMUM_LIBSODIUM_VERSION_FOR_ARGON2
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:11: warning: previous definition of MINIMUM_LIBSODIUM_VERSION_FOR_ARGON2 was here
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:12: warning: already initialized constant RbNaCl::Sodium::Version::MINIMUM_LIBSODIUM_VERSION_FOR_ARGON2ID
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:12: warning: previous definition of MINIMUM_LIBSODIUM_VERSION_FOR_ARGON2ID was here
Traceback (most recent call last):
	2: from -e:1:in `<main>'
	1: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- rbnacl (LoadError)
	15: from -e:1:in `<main>'
	14: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:156:in `require'
	13: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:168:in `rescue in require'
	12: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:168:in `require'
	11: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl.rb:12:in `<top (required)>'
	10: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
	 9: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
	 8: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:6:in `<top (required)>'
	 7: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:7:in `<module:RbNaCl>'
	 6: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:9:in `<module:Sodium>'
	 5: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:14:in `<module:Version>'
	 4: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:14:in `extend'
	 3: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium.rb:11:in `extended'
	 2: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/ffi-1.14.2/lib/ffi/library.rb:99:in `ffi_lib'
	 1: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/ffi-1.14.2/lib/ffi/library.rb:99:in `map'
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/ffi-1.14.2/lib/ffi/library.rb:145:in `block in ffi_lib': Could not open library 'sodium': dlopen(sodium, 5): image not found. (LoadError)
Could not open library 'libsodium.dylib': dlopen(libsodium.dylib, 5): image not found.
Could not open library 'libsodium.so.18': dlopen(libsodium.so.18, 5): image not found.
Could not open library 'libsodium.so.18.dylib': dlopen(libsodium.so.18.dylib, 5): image not found.
Could not open library 'libsodium.so.23': dlopen(libsodium.so.23, 5): image not found.
Could not open library 'libsodium.so.23.dylib': dlopen(libsodium.so.23.dylib, 5): image not found

While at first, this looks like I don't have libsodium installed, I want to point out that I do have it installed via brew.

ole@Oles-MBP-2 test % brew test libsodium          
==> Testing libsodium
==> /usr/bin/clang test.c -I/opt/homebrew/Cellar/libsodium/1.0.18_1/include -L/opt/homebrew/Cellar/libsodium/1.0.18_1/lib -lsodium -o test
==> ./test

But it seems it just does not pick it up. I've started my investigations over here before: RubyCrypto/rbnacl#216

I use asdf which uses ruby-build to build my ruby:

ole@Oles-MBP-2 test % ruby -v                      
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [arm64-darwin20]

Any pointers or workarounds would be appreciated 👏

[elja]

Hello @OleMchls,

I think I'm having same issues on my fresh new macbook air m1. I bought it few days ago for my wife and wanted to try if one of my projects will work well natively on ARM without any Rosseta magic.

I installed homebrew then installed rbenv. I figured that on ARM homebrew installation directory is now /opt/homebrew and all libraries are linked into the /opt/homebrew/lib. Then I installed 2.7.2 ruby ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [arm64-darwin20]. After I installed other software like 'glib' and 'vips' and 'postgresql' and did bundle install without any issues.

However after I wanted to start bin/rails c I've got an error:
Could not open library 'libglib-2.0.0.dylib': dlopen(libglib-2.0.0.dylib, 5): image not found (LoadError)

If I do ls /opt/homebrew/lib I can see that required libraries are there. Then I tried to find a good way to make ffi to look into the /opt/homebrew/lib for the libraries. I tried to set LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/DYLD_FALLBACK_LIBRARY_PATH to look into /opt/homebrew/lib but it looks like MacOS and dlopen just ignores it, or maybe I'm doing it wrong (doing it like export DYLD_LIBRARY_PATH=/opt/homebrew/lib/)

!!!!

If you open ffi code you will see that it's trying to resolve libraries that failed to load via dlopen with the following code

path = ['/usr/lib/','/usr/local/lib/','/opt/local/lib/'].find do |pth|
   File.exist?(pth + libname)
end

So I ended up with just linking /opt/homebrew/lib to /opt/local/lib. After doing that my rails console started without any issues. I know this workaround is not perfect but I already spent 4+ hours on this stuff and not really sure if its homebrew or ffi issue.

!!!

[superscott]

Can confirm the PR @OleMchls has open #882 fixes this issue.

[jasonfb]

also if you happen to have gotten here and upgrading ffi to the fixed version didn't fix your issue, try

brew install libsodium
(doh!)

[southwolf]

Should we at least put '/opt/local/lib/' before '/usr/local/lib/' to search M1 native libs first?
In apple silicon machines, when both versions are installed, the native arm64 libs ('/opt/local/lib/') should be the first option, and x64 ('/usr/local/lib/') as a failsafe fallback only when arm64 version is not available.

In my case: I've installed both x64 & arm64 version of glib, but ffi found x64 and then stopped, giving up to try /opt ones

While placing /opt before /usr/local

[eregon]

Some thoughts on this, and seeing many PRs trying to solve it: #965 by @dzprofessionerd, #936 by @southwolf and #941 by @davidperched.

The idea of a search path is we should search to find an existing file for which dlopen() succeeds, and keeps trying other paths otherwise.
The current logic seems to only look into the first match in this list of directories, because after the first absolute path try, unless libname.start_with?("/") will skip it. Fixing that would be best and most reliable, then the order would only matter if the library is of the right architecture and multiple times in the search path.

The second thing is why does Homebrew on darwin-aarch64 doesn't add its libraries on the default library search path, such that dlopen("libfoo.dylib") would just find it? Does anyone know that? That would be the expected behavior for any system package manager at least on Linux, it would just work because of using the system library search path.

[eregon]

And here is an issue for Homebrew as to why we need to special-case darwin-arm64 and why it does not just work using the system library search path: Homebrew/brew#13481

[eregon]

Let's reopen this as it's only solved if there is no Rosetta2 installation, if there is and libraries exist for both arch's it doesn't work.