Skip to content

Tensorflow build from sources fails #9697

@rajivkapoor

Description

@rajivkapoor

System information

Tensorflow build from TF 1.1 sources cloned from GIT fails. This was working a couple of days ago on the same machine.

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
    Ubuntu 16.04, kernel 4.4.0-75-generic
    Hardware: Skylake server, nVidia P4

  • TensorFlow installed from (source or binary):
    Source

  • TensorFlow version (use command below):
    I believe is 1.1-rc2 ... cannot get it from TF itself because unable to build in the first place

  • Bazel version (if compiling from source):
    Build label: 0.4.5
    Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
    Build time: Thu Mar 16 12:19:38 2017 (1489666778)
    Build timestamp: 1489666778
    Build timestamp as int: 1489666778

  • CUDA/cuDNN version:
    CUDA 8.0
    cuDNN 5.1.10

  • GPU model and memory:
    nVidia P4, 8GB

  • Exact command to reproduce:
    Followed exactly the steps to build from source from - https://www.tensorflow.org/versions/master/install/install_sources

Did the following:
./configure - used all defaults (see attached -
configure_nocuda.txt
)

bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package
OR
bazel build --verbose_failures --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

Both fail with the same issue (see below for the non cuda build)

Output from the environment collection script is attached -
tf_env.txt

Describe the problem

Build fails. I have tried it with --config=cuda also - same issue.

Source code / logs

Here are the first couple of errors

System information

Tensorflow build from TF 1.1 sources cloned from GIT fails. This was working a couple of days ago on the same machine.

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
    Ubuntu 16.04, kernel 4.4.0-75-generic
    Hardware: Skylake server, nVidia P4

  • TensorFlow installed from (source or binary):
    Source

  • TensorFlow version (use command below):
    I believe is 1.1-rc2 ... cannot get it from TF itself because unable to build in the first place

  • Bazel version (if compiling from source):
    Build label: 0.4.5
    Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
    Build time: Thu Mar 16 12:19:38 2017 (1489666778)
    Build timestamp: 1489666778
    Build timestamp as int: 1489666778

  • CUDA/cuDNN version:
    CUDA 8.0
    cuDNN 5.1.10

  • GPU model and memory:
    nVidia P4, 8GB

  • Exact command to reproduce:
    Followed exactly the steps to build from source from - https://www.tensorflow.org/versions/master/install/install_sources

Did the following:
./configure - used all defaults (see attached -
configure_nocuda.txt
)

bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package
OR
bazel build --verbose_failures --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

Both fail with the same issue (see below for the non cuda build)

Output from the environment collection script is attached -
tf_env.txt

Describe the problem

Build fails. I have tried it with --config=cuda also - same issue.

Source code / logs

Here are the first couple of errors in bold from a non-cuda build (compete build log is attached -
tf_build_nocuda.txt):

ERROR: /home/rajka/tensorflow/tensorflow/core/kernels/BUILD:2093:1: C++ compilation of rule '//tensorflow/core/kernels:self_adjoint_eig_v2_op' failed: gcc failed: error executing command
(cd /home/rajka/.cache/bazel/bazel_rajka/28f0a9835f793d5627ca9486394f31f2/execroot/tensorflow &&
exec env -
PYTHON_BIN_PATH=/usr/bin/python
PYTHON_LIB_PATH=/usr/local/lib/python2.7/dist-packages
TF_NEED_CUDA=0
TF_NEED_OPENCL=0
/usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-march=native' '-march=native' '-std=c++0x' '-march=native' '-march=native' -MD -MF bazel-out/local-opt/bin/tensorflow/core/kernels/objs/self_adjoint_eig_v2_op/tensorflow/core/kernels/self_adjoint_eig_v2_op.pic.d '-frandom-seed=bazel-out/local-opt/bin/tensorflow/core/kernels/objs/self_adjoint_eig_v2_op/tensorflow/core/kernels/self_adjoint_eig_v2_op.pic.o' -fPIC -DEIGEN_MPL2_ONLY -DTENSORFLOW_USE_JEMALLOC -DSNAPPY -iquote . -iquote bazel-out/local-opt/genfiles -iquote external/jemalloc -iquote bazel-out/local-opt/genfiles/external/jemalloc -iquote external/bazel_tools -iquote bazel-out/local-opt/genfiles/external/bazel_tools -iquote external/protobuf -iquote bazel-out/local-opt/genfiles/external/protobuf -iquote external/eigen_archive -iquote bazel-out/local-opt/genfiles/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/local-opt/genfiles/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/local-opt/genfiles/external/gif_archive -iquote external/jpeg -iquote bazel-out/local-opt/genfiles/external/jpeg -iquote external/com_googlesource_code_re2 -iquote bazel-out/local-opt/genfiles/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/local-opt/genfiles/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/local-opt/genfiles/external/fft2d -iquote external/highwayhash -iquote bazel-out/local-opt/genfiles/external/highwayhash -iquote external/png_archive -iquote bazel-out/local-opt/genfiles/external/png_archive -iquote external/zlib_archive -iquote bazel-out/local-opt/genfiles/external/zlib_archive -iquote external/snappy -iquote bazel-out/local-opt/genfiles/external/snappy -isystem external/jemalloc/include -isystem bazel-out/local-opt/genfiles/external/jemalloc/include -isystem external/bazel_tools/tools/cpp/gcc3 -isystem external/protobuf/src -isystem bazel-out/local-opt/genfiles/external/protobuf/src -isystem external/eigen_archive -isystem bazel-out/local-opt/genfiles/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/local-opt/genfiles/external/gif_archive/lib -isystem external/farmhash_archive/src -isystem bazel-out/local-opt/genfiles/external/farmhash_archive/src -isystem external/png_archive -isystem bazel-out/local-opt/genfiles/external/png_archive -isystem external/zlib_archive -isystem bazel-out/local-opt/genfiles/external/zlib_archive -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare -fno-exceptions -msse3 -pthread -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE="redacted"' '-D__TIMESTAMP
_="redacted"' '-D__TIME__="redacted"' -c tensorflow/core/kernels/self_adjoint_eig_v2_op.cc -o bazel-out/local-opt/bin/tensorflow/core/kernels/_objs/self_adjoint_eig_v2_op/tensorflow/core/kernels/self_adjoint_eig_v2_op.pic.o): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.

... snip snip ...

tensorflow/core/kernels/self_adjoint_eig_v2_op.cc:95:1: required from here
external/eigen_archive/Eigen/src/Core/util/BlasUtil.h:63:74: error: no type named 'ReturnType' in 'struct Eigen::ScalarBinaryOpTraits<__vector(8) double, std::complex, Eigen::internal::scalar_product_op<__vector(8) double, std::complex > >'
typedef typename ScalarBinaryOpTraits<LhsScalar,RhsScalar>::ReturnType Scalar;

^
In file included from external/eigen_archive/Eigen/Jacobi:27:0,
from external/eigen_archive/Eigen/Eigenvalues:16,
from ./third_party/eigen3/Eigen/Eigenvalues:1,
from tensorflow/core/kernels/self_adjoint_eig_v2_op.cc:19:
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h: In instantiation of 'void Eigen::internal::apply_rotation_in_the_plane(Eigen::DenseBase&, Eigen::DenseBase&, const Eigen::JacobiRotation&) [with VectorX = Eigen::Block<Eigen::Map<Eigen::Matrix<std::complex, -1, -1>, 0, Eigen::Stride<0, 0> >, -1, 1, true>; VectorY = Eigen::Block<Eigen::Map<Eigen::Matrix<std::complex, -1, -1>, 0, Eigen::Stride<0, 0> >, -1, 1, true>; OtherScalar = double]':
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:297:40: required from 'void Eigen::MatrixBase::applyOnTheRight(Eigen::Index, Eigen::Index, const Eigen::JacobiRotation&) [with OtherScalar = double; Derived = Eigen::Map<Eigen::Matrix<std::complex, -1, -1>, 0, Eigen::Stride<0, 0> >; Eigen::Index = long int]'
external/eigen_archive/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:861:7: required from 'void Eigen::internal::tridiagonal_qr_step(RealScalar*, RealScalar*, Index, Index, Scalar*, Index) [with int StorageOrder = 0; RealScalar = double; Scalar = std::complex; Index = long int]'
external/eigen_archive/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:520:87: required from 'Eigen::ComputationInfo Eigen::internal::computeFromTridiagonal_impl(DiagType&, SubDiagType&, Eigen::Index, bool, MatrixType&) [with MatrixType = Eigen::Matrix<std::complex, -1, -1>; DiagType = Eigen::Matrix<double, -1, 1>; SubDiagType = Eigen::Matrix<double, -1, 1>; Eigen::Index = long int]'
external/eigen_archive/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:439:49: required from 'Eigen::SelfAdjointEigenSolver& Eigen::SelfAdjointEigenSolver<_MatrixType>::compute(const Eigen::EigenBase&, int) [with InputType = Eigen::Map<const Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>, 0, Eigen::Stride<0, 0> >; _MatrixType = Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>]'
external/eigen_archive/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:168:14: required from 'Eigen::SelfAdjointEigenSolver<_MatrixType>::SelfAdjointEigenSolver(const Eigen::EigenBase&, int) [with InputType = Eigen::Map<const Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>, 0, Eigen::Stride<0, 0> >; _MatrixType = Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>]'
tensorflow/core/kernels/self_adjoint_eig_v2_op.cc:66:73: required from 'void tensorflow::SelfAdjointEigV2Op::ComputeMatrix(tensorflow::OpKernelContext*, const ConstMatrixMaps&, tensorflow::SelfAdjointEigV2Op::MatrixMaps*) [with Scalar = std::complex; tensorflow::SelfAdjointEigV2Op::ConstMatrixMaps = tensorflow::gtl::InlinedVector<Eigen::Map<const Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>, 0, Eigen::Stride<0, 0> >, 4>; tensorflow::SelfAdjointEigV2Op::MatrixMaps = tensorflow::gtl::InlinedVector<Eigen::Map<Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>, 0, Eigen::Stride<0, 0> >, 4>]'
tensorflow/core/kernels/self_adjoint_eig_v2_op.cc:95:1: required from here
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:359:24: error: 'struct Eigen::internal::conj_helper<__vector(8) double, std::complex, false, false>' has no member named 'pmul'
pstore(px, padd(pm.pmul(pc,xi),pcj.pmul(ps,yi)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:359:24: error: 'struct Eigen::internal::conj_helper<__vector(8) double, std::complex, false, false>' has no member named 'pmul'

in bold from a non-cuda build (compete build log is attached -
tf_build_nocuda.txt):

ERROR: /home/rajka/tensorflow/tensorflow/core/kernels/BUILD:2093:1: C++ compilation of rule '//tensorflow/core/kernels:self_adjoint_eig_v2_op' failed: gcc failed: error executing command
(cd /home/rajka/.cache/bazel/bazel_rajka/28f0a9835f793d5627ca9486394f31f2/execroot/tensorflow &&
exec env -
PYTHON_BIN_PATH=/usr/bin/python
PYTHON_LIB_PATH=/usr/local/lib/python2.7/dist-packages
TF_NEED_CUDA=0
TF_NEED_OPENCL=0
/usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-march=native' '-march=native' '-std=c++0x' '-march=native' '-march=native' -MD -MF bazel-out/local-opt/bin/tensorflow/core/kernels/objs/self_adjoint_eig_v2_op/tensorflow/core/kernels/self_adjoint_eig_v2_op.pic.d '-frandom-seed=bazel-out/local-opt/bin/tensorflow/core/kernels/objs/self_adjoint_eig_v2_op/tensorflow/core/kernels/self_adjoint_eig_v2_op.pic.o' -fPIC -DEIGEN_MPL2_ONLY -DTENSORFLOW_USE_JEMALLOC -DSNAPPY -iquote . -iquote bazel-out/local-opt/genfiles -iquote external/jemalloc -iquote bazel-out/local-opt/genfiles/external/jemalloc -iquote external/bazel_tools -iquote bazel-out/local-opt/genfiles/external/bazel_tools -iquote external/protobuf -iquote bazel-out/local-opt/genfiles/external/protobuf -iquote external/eigen_archive -iquote bazel-out/local-opt/genfiles/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/local-opt/genfiles/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/local-opt/genfiles/external/gif_archive -iquote external/jpeg -iquote bazel-out/local-opt/genfiles/external/jpeg -iquote external/com_googlesource_code_re2 -iquote bazel-out/local-opt/genfiles/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/local-opt/genfiles/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/local-opt/genfiles/external/fft2d -iquote external/highwayhash -iquote bazel-out/local-opt/genfiles/external/highwayhash -iquote external/png_archive -iquote bazel-out/local-opt/genfiles/external/png_archive -iquote external/zlib_archive -iquote bazel-out/local-opt/genfiles/external/zlib_archive -iquote external/snappy -iquote bazel-out/local-opt/genfiles/external/snappy -isystem external/jemalloc/include -isystem bazel-out/local-opt/genfiles/external/jemalloc/include -isystem external/bazel_tools/tools/cpp/gcc3 -isystem external/protobuf/src -isystem bazel-out/local-opt/genfiles/external/protobuf/src -isystem external/eigen_archive -isystem bazel-out/local-opt/genfiles/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/local-opt/genfiles/external/gif_archive/lib -isystem external/farmhash_archive/src -isystem bazel-out/local-opt/genfiles/external/farmhash_archive/src -isystem external/png_archive -isystem bazel-out/local-opt/genfiles/external/png_archive -isystem external/zlib_archive -isystem bazel-out/local-opt/genfiles/external/zlib_archive -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare -fno-exceptions -msse3 -pthread -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE="redacted"' '-D__TIMESTAMP
_="redacted"' '-D__TIME__="redacted"' -c tensorflow/core/kernels/self_adjoint_eig_v2_op.cc -o bazel-out/local-opt/bin/tensorflow/core/kernels/_objs/self_adjoint_eig_v2_op/tensorflow/core/kernels/self_adjoint_eig_v2_op.pic.o): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.

... snip snip ...

tensorflow/core/kernels/self_adjoint_eig_v2_op.cc:95:1: required from here
external/eigen_archive/Eigen/src/Core/util/BlasUtil.h:63:74: error: no type named 'ReturnType' in 'struct Eigen::ScalarBinaryOpTraits<__vector(8) double, std::complex, Eigen::internal::scalar_product_op<__vector(8) double, std::complex > >'
typedef typename ScalarBinaryOpTraits<LhsScalar,RhsScalar>::ReturnType Scalar;

^
In file included from external/eigen_archive/Eigen/Jacobi:27:0,
from external/eigen_archive/Eigen/Eigenvalues:16,
from ./third_party/eigen3/Eigen/Eigenvalues:1,
from tensorflow/core/kernels/self_adjoint_eig_v2_op.cc:19:
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h: In instantiation of 'void Eigen::internal::apply_rotation_in_the_plane(Eigen::DenseBase&, Eigen::DenseBase&, const Eigen::JacobiRotation&) [with VectorX = Eigen::Block<Eigen::Map<Eigen::Matrix<std::complex, -1, -1>, 0, Eigen::Stride<0, 0> >, -1, 1, true>; VectorY = Eigen::Block<Eigen::Map<Eigen::Matrix<std::complex, -1, -1>, 0, Eigen::Stride<0, 0> >, -1, 1, true>; OtherScalar = double]':
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:297:40: required from 'void Eigen::MatrixBase::applyOnTheRight(Eigen::Index, Eigen::Index, const Eigen::JacobiRotation&) [with OtherScalar = double; Derived = Eigen::Map<Eigen::Matrix<std::complex, -1, -1>, 0, Eigen::Stride<0, 0> >; Eigen::Index = long int]'
external/eigen_archive/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:861:7: required from 'void Eigen::internal::tridiagonal_qr_step(RealScalar*, RealScalar*, Index, Index, Scalar*, Index) [with int StorageOrder = 0; RealScalar = double; Scalar = std::complex; Index = long int]'
external/eigen_archive/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:520:87: required from 'Eigen::ComputationInfo Eigen::internal::computeFromTridiagonal_impl(DiagType&, SubDiagType&, Eigen::Index, bool, MatrixType&) [with MatrixType = Eigen::Matrix<std::complex, -1, -1>; DiagType = Eigen::Matrix<double, -1, 1>; SubDiagType = Eigen::Matrix<double, -1, 1>; Eigen::Index = long int]'
external/eigen_archive/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:439:49: required from 'Eigen::SelfAdjointEigenSolver& Eigen::SelfAdjointEigenSolver<_MatrixType>::compute(const Eigen::EigenBase&, int) [with InputType = Eigen::Map<const Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>, 0, Eigen::Stride<0, 0> >; _MatrixType = Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>]'
external/eigen_archive/Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h:168:14: required from 'Eigen::SelfAdjointEigenSolver<_MatrixType>::SelfAdjointEigenSolver(const Eigen::EigenBase&, int) [with InputType = Eigen::Map<const Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>, 0, Eigen::Stride<0, 0> >; _MatrixType = Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>]'
tensorflow/core/kernels/self_adjoint_eig_v2_op.cc:66:73: required from 'void tensorflow::SelfAdjointEigV2Op::ComputeMatrix(tensorflow::OpKernelContext*, const ConstMatrixMaps&, tensorflow::SelfAdjointEigV2Op::MatrixMaps*) [with Scalar = std::complex; tensorflow::SelfAdjointEigV2Op::ConstMatrixMaps = tensorflow::gtl::InlinedVector<Eigen::Map<const Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>, 0, Eigen::Stride<0, 0> >, 4>; tensorflow::SelfAdjointEigV2Op::MatrixMaps = tensorflow::gtl::InlinedVector<Eigen::Map<Eigen::Matrix<std::complex, -1, -1, 1, -1, -1>, 0, Eigen::Stride<0, 0> >, 4>]'
tensorflow/core/kernels/self_adjoint_eig_v2_op.cc:95:1: required from here
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:359:24: error: 'struct Eigen::internal::conj_helper<__vector(8) double, std::complex, false, false>' has no member named 'pmul'
pstore(px, padd(pm.pmul(pc,xi),pcj.pmul(ps,yi)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:359:24: error: 'struct Eigen::internal::conj_helper<__vector(8) double, std::complex, false, false>' has no member named 'pmul'

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions