AddDescriptors is not thread safe, but multi-thread access may occur when the SO is opened through dlopen

[xuquanwei2019]

What version of protobuf and what language are you using?
Version: 29.4
Language: C++

According to the code, the AddDescriptors function is invoked only before the main function. Therefore, the AssignDescriptorImpl invoking point is protected by mutex. However, when the dlopen function is used, multiple threads invoke AddDescriptors and AssignDescriptorImpl, causing an exception coredump.

stack of dlopen
#0 google::protobuf::(anonymous namespace)::AddDescriptors (table=0x7f20dc5bf180 <descriptor_table_sttestservice_2eproto>)
#1 0x00007f21d0d9d55c in google::protobuf::internal::AddDescriptorsRunner::AddDescriptorsRunner (this=0x7f20dc5c80c4 <dynamic_init_dummy_sttestservice_2eproto>, table=0x7f20dc5bf180 <descriptor_table_sttestservice_2eproto>)
#2 0x00007f20dc3731d9 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=102) at /usr1/ONIP_Agilit/test/pbfile/sttestservice.pb.cc:146
#3 0x00007f20dc373204 in _GLOBAL__sub_I.00102_sttestservice.pb.cc(void) () at /usr1/ONIP_Agilit/dsf_cpp/thirdparty/protobuf/include/google/protobuf/port_undef.inc:179
#4 0x00007f21d6e5b09a in call_init.part () from /lib64/ld-linux-x86-64.so.2
#5 0x00007f21d6e5b183 in _dl_init () from /lib64/ld-linux-x86-64.so.2
#6 0x00007f21d6e5f39d in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#7 0x00007f21d6e5af54 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#8 0x00007f21d6e5ea7b in _dl_open () from /lib64/ld-linux-x86-64.so.2

stack of coredump
libc.so.6!raise
libc.so.6!abort
libc.so.6!__assert_fail_base
libc.so.6!__assert_fail
libdsf.so!absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::emplace_at<google::protobuf::internal::DescriptorTable const* const&>(absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> > * const this, size_t i, args#0) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\container\internal\raw_hash_set.h:2702)
libdsf.so!absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::EmplaceDecomposable::operator()<google::protobuf::internal::DescriptorTable const*, google::protobuf::internal::DescriptorTable const* const&>(const absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::EmplaceDecomposable * const this, const google::protobuf::internal::DescriptorTable * const& key, args#0) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\container\internal\raw_hash_set.h:2461)
libdsf.so!absl::lts_20230802::container_internal::DecomposeValue<absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::EmplaceDecomposable, google::protobuf::internal::DescriptorTable const* const&>(absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::EmplaceDecomposable && f, const google::protobuf::internal::DescriptorTable * const& arg) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\container\internal\container_memory.h:216)
libdsf.so!absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>::apply<absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::EmplaceDecomposable, google::protobuf::internal::DescriptorTable const* const&>(absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::EmplaceDecomposable && f, args#0) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\container\flat_hash_set.h:488)
libdsf.so!absl::lts_20230802::container_internal::hash_policy_traits<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, void>::apply<absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::EmplaceDecomposable, google::protobuf::internal::DescriptorTable const* const&>(absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::EmplaceDecomposable && f, ts#0) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\container\internal\hash_policy_traits.h:134)
libdsf.so!absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::emplace<google::protobuf::internal::DescriptorTable const* const&>(absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> > * const this, args#0) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\container\internal\raw_hash_set.h:2065)
libdsf.so!absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> >::insert<google::protobuf::internal::DescriptorTable const*>(absl::lts_20230802::container_internal::raw_hash_set<absl::lts_20230802::container_internal::FlatHashSetPolicy<google::protobuf::internal::DescriptorTable const*>, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameHash, google::protobuf::(anonymous namespace)::GeneratedMessageFactory::DescriptorByNameEq, std::allocator<google::protobuf::internal::DescriptorTable const*> > * const this, const google::protobuf::internal::DescriptorTable * const& value) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\container\internal\raw_hash_set.h:1981)
libdsf.so!google::protobuf::(anonymous namespace)::GeneratedMessageFactory::RegisterFile(google::protobuf::(anonymous namespace)::GeneratedMessageFactory * const this, const google::protobuf::internal::DescriptorTable * table) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\message.cc:277)
libdsf.so!google::protobuf::MessageFactory::InternalRegisterGeneratedFile(const google::protobuf::internal::DescriptorTable * table) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\message.cc:347)
libdsf.so!google::protobuf::(anonymous namespace)::AddDescriptorsImpl(const google::protobuf::internal::DescriptorTable * table) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\generated_message_reflection.cc:3674)
libdsf.so!google::protobuf::(anonymous namespace)::AddDescriptors(const google::protobuf::internal::DescriptorTable * table) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\generated_message_reflection.cc:3684)
libdsf.so!google::protobuf::(anonymous namespace)::AssignDescriptorsImpl(const google::protobuf::internal::DescriptorTable * table, bool eager) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\generated_message_reflection.cc:3598)
libdsf.so!google::protobuf::internal::<lambda()>::operator()(void) const(const google::protobuf::internal::<lambda()> * const __closure) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\generated_message_reflection.cc:3707)
libdsf.so!absl::lts_20230802::base_internal::Callable::Invoke<google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable* ()(), absl::lts_20230802::once_flag, const google::protobuf::Metadata&)::<lambda()> >(google::protobuf::internal::<lambda()> &&)(google::protobuf::internal::<lambda()> && f) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\base\internal\invoke.h:185)
libdsf.so!absl::lts_20230802::base_internal::invoke<google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable* ()(), absl::lts_20230802::once_flag, const google::protobuf::Metadata&)::<lambda()> >(google::protobuf::internal::<lambda()> &&)(google::protobuf::internal::<lambda()> && f) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\base\internal\invoke.h:212)
libdsf.so!absl::lts_20230802::base_internal::CallOnceImpl<google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable* ()(), absl::lts_20230802::once_flag, const google::protobuf::Metadata&)::<lambda()> >(std::atomic , absl::lts_20230802::base_internal::SchedulingMode, google::protobuf::internal::<lambda()> &&)(std::atomic * control, absl::lts_20230802::base_internal::SchedulingMode scheduling_mode, google::protobuf::internal::<lambda()> && fn) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\base\call_once.h:178)
libdsf.so!absl::lts_20230802::call_once<google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable ()(), absl::lts_20230802::once_flag, const google::protobuf::Metadata&)::<lambda()> >(absl::lts_20230802::once_flag &, google::protobuf::internal::<lambda()> &&)(absl::lts_20230802::once_flag & flag, google::protobuf::internal::<lambda()> && fn) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\base\call_once.h:210)
libdsf.so!google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable ()(void) table, absl::lts_20230802::once_flag * once, const google::protobuf::Metadata & metadata) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\generated_message_reflection.cc:3704)
libdsf.so!google::protobuf::DescriptorProto::GetMetadata(const google::protobuf::DescriptorProto * const this) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\descriptor.pb.cc:4132)
libdsf.so!google::protobuf::DescriptorProto::GetDescriptor() (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\descriptor.pb.h:9939)
libdsf.so!google::protobuf::DescriptorProto::descriptor() (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\descriptor.pb.h:9936)
libdsf.so!google::protobuf::DescriptorPool::generated_pool() (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\descriptor.cc:2091)
libdsf.so!google::protobuf::(anonymous namespace)::MaybeInitializeLazyDescriptors(const google::protobuf::internal::DescriptorTable * table) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\generated_message_reflection.cc:3655)
libdsf.so!google::protobuf::internal::<lambda()>::operator()(void) const(const google::protobuf::internal::<lambda()> * const __closure) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\generated_message_reflection.cc:3706)
libdsf.so!absl::lts_20230802::base_internal::Callable::Invoke<google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable* ()(), absl::lts_20230802::once_flag, const google::protobuf::Metadata&)::<lambda()> >(google::protobuf::internal::<lambda()> &&)(google::protobuf::internal::<lambda()> && f) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\base\internal\invoke.h:185)
libdsf.so!absl::lts_20230802::base_internal::invoke<google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable* ()(), absl::lts_20230802::once_flag, const google::protobuf::Metadata&)::<lambda()> >(google::protobuf::internal::<lambda()> &&)(google::protobuf::internal::<lambda()> && f) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\base\internal\invoke.h:212)
libdsf.so!absl::lts_20230802::base_internal::CallOnceImpl<google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable* ()(), absl::lts_20230802::once_flag, const google::protobuf::Metadata&)::<lambda()> >(std::atomic , absl::lts_20230802::base_internal::SchedulingMode, google::protobuf::internal::<lambda()> &&)(std::atomic * control, absl::lts_20230802::base_internal::SchedulingMode scheduling_mode, google::protobuf::internal::<lambda()> && fn) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\base\call_once.h:178)
libdsf.so!absl::lts_20230802::call_once<google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable ()(), absl::lts_20230802::once_flag, const google::protobuf::Metadata&)::<lambda()> >(absl::lts_20230802::once_flag &, google::protobuf::internal::<lambda()> &&)(absl::lts_20230802::once_flag & flag, google::protobuf::internal::<lambda()> && fn) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\third_party\abseil-cpp\absl\base\call_once.h:210)
libdsf.so!google::protobuf::internal::AssignDescriptors(const google::protobuf::internal::DescriptorTable ()(void) table, absl::lts_20230802::once_flag * once, const google::protobuf::Metadata & metadata) (\usr1\ONIP_Agilit\build\cmake\cmake\ThirdParty\protobuf\protobuf_external-prefix\src\protobuf_external\src\google\protobuf\generated_message_reflection.cc:3704)
libDSFApi.so!google::protobuf::Message::GetDescriptor(const google::protobuf::Message * const this) (\usr1\ONIP_Agilit\dsf_cpp\thirdparty\protobuf\include\google\protobuf\message.h:354)
libDSFApi.so!(anonymous namespace)::GetPbField(const google::protobuf::Message & msg, const char * name, google::protobuf::FieldDescriptor::Type type) (\usr1\ONIP_Agilit\src\dsfcpp\common\msg\pb2json.cpp:386)
libDSFApi.so!(anonymous namespace)::GetInt32Field(const char * name, const google::protobuf::Message & msg) (\usr1\ONIP_Agilit\src\dsfcpp\common\msg\pb2json.cpp:411)
libDSFApi.so!GetInt32FromPb(const google::protobuf::Message & msg, const char * pszField, int & data) (\usr1\ONIP_Agilit\src\dsfcpp\common\msg\pb2json.cpp:577)
libDSFApi.so!CPBMsgHelper::GetInt32FromPb(const dsf::PbMessage & pb, const char * pszField, mpr_int32_t & data) (\usr1\ONIP_Agilit\src\dsfcpp\common\msg\PBMsgHelper.cpp:171)
libDSFCacheClient.so!dsfapp::SyncCallDsfWithResult(dsf::IControllerEx * controller, dsf::PbMessage * request, dsf::PbMessage * reply) (\usr1\ONIP_Agilit\src\dsfcpp\cacheservice\dsfclient\CacheClientUtils.cpp:330)
libDSFCacheClient.so!dsfapp::ClientCacheData::CcGetTableInfo(const dsfapp::ClientCacheData * const this, const std::vector<char const*, std::allocator<char const*> > & tablenames, std::vector<dsfapp::TTableInfo, std::allocatordsfapp::TTableInfo > & tables, const dsf::InetAddress & addr) (\usr1\ONIP_Agilit\src\dsfcpp\cacheservice\dsfclient\ClientCacheData.cpp:477)
libDSFCacheClient.so!dsfapp::ClientCacheData::RefreshTableInfo(dsfapp::ClientCacheData * const this, mpr_uint64_t srvAddrHash, const std::vector<char const*, std::allocator<char const*> > & tablenames) (\usr1\ONIP_Agilit\src\dsfcpp\cacheservice\dsfclient\ClientCacheData.cpp:867)
libDSFCacheClient.so!dsfapp::ClientCacheData::GetTableInfo(dsfapp::ClientCacheData * const this, dsfapp::TCacheClientHandle & handle, const std::vector<char const*, std::allocator<char const*> > & tablenames, std::vector<dsfapp::TTableInfoPS*, std::allocatordsfapp::TTableInfoPS* > & tables) (\usr1\ONIP_Agilit\src\dsfcpp\cacheservice\dsfclient\ClientCacheData.cpp:135)
libDSFCacheClient.so!dsfapp::ClientCacheData::CAutoReadTableInfoLock::CAutoReadTableInfoLock(dsfapp::ClientCacheData::CAutoReadTableInfoLock * const this, dsfapp::ClientCacheData & ccData, dsfapp::TCacheClientHandle & handle, const char * tablename, bool bIsDirect, const char * func) (\usr1\ONIP_Agilit\src\dsfcpp\cacheservice\dsfclient\ClientCacheData.cpp:973)
libDSFCacheClient.so!dsfapp::ClientCacheData::GetTableVersion(dsfapp::ClientCacheData * const this, dsfapp::TCacheClientHandle & handle, const char * tablename, unsigned int & ddlversion, unsigned long long & dmlversion, bool bIsDirect) (\usr1\ONIP_Agilit\src\dsfcpp\cacheservice\dsfclient\ClientCacheData.cpp:238)
libDSFCacheClient.so!CcXcGetTableVersion(void * ctx, const char * tablename, unsigned int & ddlversion, unsigned long long & dmlversion, bool bIsDirect) (\usr1\ONIP_Agilit\src\dsfcpp\cacheservice\dsfclient\CacheClient.cpp:772)
libcacheapi.so!BOCACHE::XcGetTableVersion0(BOCACHE::cache_handle pCacheHandle, const char * tablename, unsigned int & ddlversion, unsigned long long & dmlversion, bool bIsDirect) (\usr1\ONIP_Agilit\src\bocache\source\kernel\cache_api.cpp:2339)
libcacheapi.so!BOCACHE::XcGetTableVersion(BOCACHE::cache_handle pCacheHandle, const char * tablename, unsigned int & ddlversion, unsigned long long & dmlversion) (\usr1\ONIP_Agilit\src\bocache\source\kernel\cache_api.cpp:2362)
librouter.so!CWeightedDataCacheMgr::ReadTable(CWeightedDataCacheMgr * const this, BOCACHE::cache_handle pHandle, const char * pszTable, const char * pszIndex, CWeightedDataCache & dataCache) (\usr1\ONIP_Agilit\src\dsf\source\mech_router\WeightedDataCache.cpp:1087)
librouter.so!CWeightedDataCacheMgr::LoadDataFromCache(CWeightedDataCacheMgr * const this) (\usr1\ONIP_Agilit\src\dsf\source\mech_router\WeightedDataCache.cpp:1072)
librouter.so!CWeightedDataCacheMgr::RefreshWeightedPolicy(CWeightedDataCacheMgr * const this) (\usr1\ONIP_Agilit\src\dsf\source\mech_router\WeightedDataCache.cpp:1020)
libdsfbase.so!CTaskManager::DoTasks(CTaskManager * const this, const CTaskManager::TTasks & tasks) (\usr1\ONIP_Agilit\src\base\source\utils\TaskManagerImpl.cpp:150)
libdsfbase.so!CTaskManager::Execute(CTaskManager * const this) (\usr1\ONIP_Agilit\src\base\source\utils\TaskManagerImpl.cpp:168)
libnmgrapi.so!CNodeManagerClient::OneAction(CNodeManagerClient * const this, const CNodeClientWorker * worker, const std::chrono::milliseconds & iTimeout) (\usr1\ONIP_Agilit\src\node_manager\source\ClientAPI\nmgr_cli.cpp:234)
libnmgrapi.so!CNodeClientWorker::OneAction(CNodeClientWorker * const this) (\usr1\ONIP_Agilit\src\node_manager\source\ClientAPI\nmgr_cli.cpp:539)
libdsfbase.so!CThread::Run(CThread * const this) (\usr1\ONIP_Agilit\src\base\source\shr\thread_pool.cpp:214)
libdsfbase.so!CThread::Execute(CThread * const this) (\usr1\ONIP_Agilit\src\base\source\shr\thread_pool.cpp:259)
libdsfbase.so!CThread::ThreadProc(void * pObj) (\usr1\ONIP_Agilit\src\base\source\shr\thread_pool.cpp:280)
libdsfbase.so!ThreadRun(void * param) (\usr1\ONIP_Agilit\src\base\source\mpr\mpr_thread_unix.cpp:303)
libpthread.so.0!start_thread
libc.so.6!clone

[acozzette]

I don't know if we have this documented anywhere, but we don't support using dlopen this way unfortunately. We assume that all this initialization happens before main() and as a result, we can't handle protobuf generated code trying to register itself in our global data structures (such as the global descriptor pool) later on.

I think that if you want to dlopen a shared object containing protobuf gencode, then the only way to do it would be for that SO to have its own isolated copy of the protobuf runtime, with symbols hidden to prevent ODR violations. I have not personally tried this, but I believe it would work. You would not be able to pass protos across the SO boundary, though.

[github-actions]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

[pitrou]

I don't know if we have this documented anywhere, but we don't support using dlopen this way unfortunately. We assume that all this initialization happens before main() and as a result, we can't handle protobuf generated code trying to register itself in our global data structures (such as the global descriptor pool) later on.

Would it be possible to reconsider this? Usage of dlopen is very widespread (it's implicit when loading a Python extension) and hiding symbols can be annoyingly tricky (GNU ld version scripts are not very powerful).

[acozzette]

Unfortunately I don't think we would want to add support for this, as it seems like a tricky use case to support and one that we don't internally use.

In my experience version scripts usually don't need to be very complicated and look something like this:

VERS_1.0 {
  global:
    # whatever symbols you need to expose...

  # Hide everything else.
  local:
    *;
};

Is there some reason that your script would need to be more complicated?

[pitrou]

Is there some reason that your script would need to be more complicated?

The reason is that it's not possible (well, technically it would be possible, but...) to exhaustively list all symbols that a C++ library like Apache Arrow is supposed to expose.

So we have to use a best effort approach using wildcards. The problem is that wildcards are not powerful enough to exclude Protobuf symbols that involve Arrow types while including Arrow APIs. Is that a problem? Well, who knows.

And, besides, version scripts are not a universal feature, it's not obvious that linkers other than GNU ld support it.

[pitrou]

For the record, this seems to be hitting us when trying to build musllinux packages (the Python binary package kind for Linux with musl, such as Alpine Linux) for PyArrow:
apache/arrow#47616 (comment)

For some reason we didn't have this problem before, and we don't see this problem on manylinux (the Python binary package kind for Linux with glibc - i.e. most mainstream Linux distros).