Skip to content

Conversation

snnn
Copy link
Member

@snnn snnn commented Jul 9, 2024

Description

Motivation and Context

Our macOS pipeline are failing because of a build error in absl. However, the bug fix we need is not available in the latest ABSL release.

Here is the issue: abseil/abseil-cpp#1536
And here is the fix: abseil/abseil-cpp@779a356

GTests uses ABSL. But this ABSL target also depends on GTest. So, it is a circular dependency. We should be able to avoid that by avoid building tests for ABSL. However, the version we are using has a problem with that: it has cmake target that still depends on GTest even when testing is disabled.

I don't know why we suddenly hit this problem.

@yuslepukhin
Copy link
Member

Do we know if the internals of the classes are changing? Question is related to natvis.

@snnn
Copy link
Member Author

snnn commented Jul 9, 2024

Most likely they are changing.

@snnn
Copy link
Member Author

snnn commented Jul 10, 2024

GCC 12 seems to have a bug with the latest ABSL code.

$ /opt/rh/gcc-toolset-12/root/usr/bin/c++ -DCPUINFO_SUPPORTED_PLATFORM=1 -DEIGEN_MPL2_ONLY -DEIGEN_USE_THREADS -DNSYNC_ATOMIC_CPP11 -DONLY_C_LOCALE=0 -DONNX_ML=1 -DONNX_NAMESPACE=onnx -DONNX_USE_LITE_PROTO=1 -DORT_ENABLE_STREAM -DORT_RUN_EXTERNAL_ONNX_TESTS -DPLATFORM_POSIX -D_GNU_SOURCE -D__ONNX_NO_DOC_STRINGS -I/tmp/build5/Release/_deps/utf8_range-src -I/onnxruntime_src/include/onnxruntime -I/onnxruntime_src/include/onnxruntime/core/session -I/tmp/build5/Release/_deps/pytorch_cpuinfo-src/include -I/tmp/build5/Release/_deps/google_nsync-src/public -I/tmp/build5/Release -I/onnxruntime_src/onnxruntime -I/tmp/build5/Release/_deps/abseil_cpp-src -I/tmp/build5/Release/_deps/safeint-src -I/tmp/build5/Release/_deps/gsl-src/include -I/tmp/build5/Release/_deps/date-src/include -I/tmp/build5/Release/_deps/onnx-src -I/tmp/build5/Release/_deps/onnx-build -I/tmp/build5/Release/_deps/protobuf-src/src -I/tmp/build5/Release/_deps/flatbuffers-src/include -I/tmp/build5/Release/_deps/mp11-src/include -I/tmp/build5/Release/_deps/eigen-src -I/tmp/build5/Release/_deps/re2-src -DNDEBUG -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -O3 -pipe -fstack-clash-protection -fcf-protection -ffunction-sections -fdata-sections -Wno-restrict  -DCPUINFO_SUPPORTED -O3 -DNDEBUG -fPIC -Wall -Wextra -Wno-deprecated-copy -Wno-nonnull-compare -Werror /onnxruntime_src/slice.cc  -c
In file included from /tmp/build5/Release/_deps/abseil_cpp-src/absl/container/flat_hash_set.h:45,
                 from /onnxruntime_src/include/onnxruntime/core/common/inlined_containers.h:21,
                 from /onnxruntime_src/include/onnxruntime/core/graph/graph.h:23,
                 from /onnxruntime_src/include/onnxruntime/core/graph/graph_viewer.h:8,
                 from /onnxruntime_src/include/onnxruntime/core/framework/op_node_proto_helper.h:9,
                 from /onnxruntime_src/include/onnxruntime/core/framework/op_kernel_info.h:9,
                 from /onnxruntime_src/include/onnxruntime/core/framework/op_kernel.h:20,
                 from /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/slice.h:7,
                 from /onnxruntime_src/slice.cc:4:
/tmp/build5/Release/_deps/abseil_cpp-src/absl/container/internal/raw_hash_set.h: In function ‘onnxruntime::common::Status onnxruntime::SliceOp::PrepareForComputeHelper(const gsl::span<const long int>&, const gsl::span<const long int>&, const gsl::span<const long int>&, const gsl::span<const long int>&, PrepareForComputeMetadata&)’:
/tmp/build5/Release/_deps/abseil_cpp-src/absl/container/internal/raw_hash_set.h:1292:54: error: ‘unique_axes.onnxruntime::InlinedHashSet<long int, std::allocator<long int> >::<unnamed>.absl::flat_hash_set<long int, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::<unnamed>.absl::container_internal::raw_hash_set<absl::container_internal::FlatHashSetPolicy<long int>, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::settings_.absl::container_internal::CompressedTuple<absl::container_internal::CommonFields, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::<unnamed>.absl::container_internal::internal_compressed_tuple::CompressedTupleImpl<absl::container_internal::CompressedTuple<absl::container_internal::CommonFields, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >, std::integer_sequence<long unsigned int, 0, 1, 2, 3>, true>::<unnamed>.absl::container_internal::internal_compressed_tuple::Storage<absl::container_internal::CommonFields, 0, false>::value.absl::container_internal::CommonFields::heap_or_soo_.absl::container_internal::HeapOrSoo::heap.absl::container_internal::HeapPtrs::control’ may be used uninitialized [-Werror=maybe-uninitialized]
 1292 |     ABSL_SWISSTABLE_IGNORE_UNINITIALIZED_RETURN(heap.control);
      |                                                      ^~~~~~~
In file included from /onnxruntime_src/slice.cc:13:
/onnxruntime_src/onnxruntime/core/providers/cpu/tensor/slice_helper.h:98:11: note: ‘unique_axes’ declared here
   98 |   AxesSet unique_axes;
      |           ^~~~~~~~~~~
/tmp/build5/Release/_deps/abseil_cpp-src/absl/container/internal/raw_hash_set.h: In function ‘onnxruntime::common::Status onnxruntime::SliceOp::PrepareForComputeHelper(const gsl::span<const long int>&, const gsl::span<const long int>&, const gsl::span<const long int>&, PrepareForComputeMetadata&)’:
/tmp/build5/Release/_deps/abseil_cpp-src/absl/container/internal/raw_hash_set.h:1292:54: error: ‘unique_axes.onnxruntime::InlinedHashSet<long int, std::allocator<long int> >::<unnamed>.absl::flat_hash_set<long int, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::<unnamed>.absl::container_internal::raw_hash_set<absl::container_internal::FlatHashSetPolicy<long int>, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::settings_.absl::container_internal::CompressedTuple<absl::container_internal::CommonFields, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::<unnamed>.absl::container_internal::internal_compressed_tuple::CompressedTupleImpl<absl::container_internal::CompressedTuple<absl::container_internal::CommonFields, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >, std::integer_sequence<long unsigned int, 0, 1, 2, 3>, true>::<unnamed>.absl::container_internal::internal_compressed_tuple::Storage<absl::container_internal::CommonFields, 0, false>::value.absl::container_internal::CommonFields::heap_or_soo_.absl::container_internal::HeapOrSoo::heap.absl::container_internal::HeapPtrs::control’ may be used uninitialized [-Werror=maybe-uninitialized]
 1292 |     ABSL_SWISSTABLE_IGNORE_UNINITIALIZED_RETURN(heap.control);
      |                                                      ^~~~~~~
/onnxruntime_src/onnxruntime/core/providers/cpu/tensor/slice_helper.h:39:11: note: ‘unique_axes’ declared here
   39 |   AxesSet unique_axes;
      |           ^~~~~~~~~~~
cc1plus: all warnings being treated as errors

GCC 13 doesn't have this issue.

@snnn snnn marked this pull request as ready for review July 10, 2024 15:41
@snnn snnn requested a review from a team July 10, 2024 15:41
@snnn snnn requested a review from a team as a code owner July 10, 2024 15:47
@snnn snnn merged commit 8749fa3 into main Jul 10, 2024
@snnn snnn deleted the snnn/update_absl2 branch July 10, 2024 18:14
sumitsays pushed a commit that referenced this pull request Aug 8, 2024
Our macOS pipeline are failing because of a build error in absl.
However, the bug fix we need is not available in the latest ABSL
release.

Here  is the issue: abseil/abseil-cpp#1536
And here is the fix:
abseil/abseil-cpp@779a356

GTests uses ABSL. But this ABSL target also depends on GTest. So, it is
a circular dependency. We should be able to avoid that by avoid building
tests for ABSL. However, the version we are using has a problem with
that: it has cmake target that still depends on GTest even when testing
is disabled.

It's strange that we suddenly hit this problem and it only happens on macOS.
prathikr pushed a commit that referenced this pull request Aug 8, 2024
Our macOS pipeline are failing because of a build error in absl.
However, the bug fix we need is not available in the latest ABSL
release.

Here  is the issue: abseil/abseil-cpp#1536
And here is the fix:
abseil/abseil-cpp@779a356

GTests uses ABSL. But this ABSL target also depends on GTest. So, it is
a circular dependency. We should be able to avoid that by avoid building
tests for ABSL. However, the version we are using has a problem with
that: it has cmake target that still depends on GTest even when testing
is disabled.

It's strange that we suddenly hit this problem and it only happens on macOS.
sumitsays added a commit that referenced this pull request Aug 12, 2024
…1670)

### Description
This change cherry-picks 2 Pad fusion optimization:
#21640 and
#21556.

It also has to cherry-pick 2 extra changes to unblock pipeline and
dependency failure: #21300
and #21662 (didn't include
test which are part of 1.18.1 payload).

Also uploaded new version of
[onnxruntime_build_dependencies:10.177](https://dev.azure.com/onnxruntime/onnxruntime/_artifacts/feed/onnxruntime/UPack/onnxruntime_build_dependencies/overview/1.0.177)
and updated the same in `download-deps.yml`.

Additionally it also updates DML binary to 1.15.1.



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: Changming Sun <[email protected]>
Co-authored-by: Tianlei Wu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants