error: pshufb is not supported on generic64.aes

[yangming]

Hi, I compiled the recent git version of ncbi-vdb, including ver 2.8.2, in my arch linux server (updated recently), it gave this error when testing after make. how could i fix this. thx in advance.

gcc -c -DNDEBUG -m64 -DLINUX -DUNIX -D_GNU_SOURCE -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPKGNAME=linux64 -D_ARCH_BITS=_SIZEOF_POINTER_*CHAR_BIT -DLIBPREFIX=lib
-DSHLIBEXT=so -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/override -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/linux -I/home/
eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/unix -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto -I/home/eastasia/softhouse/biotools/git_s
vn/ncbi-vdb/interfaces -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/cc/gcc/x86_64 -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/cc/
gcc -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/os/linux -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/os/unix -I/home/eastasia/so
fthouse/biotools/git_svn/ncbi-vdb/interfaces/ext -I/usr/include -I. -MD -o aes-ncbi.aes-ni.pic.o -fPIC -O3 -Wno-variadic-macros -fno-strict-aliasing -U_FORTIFY_SOURCE -
D_FORTIFY_SOURCE=1 -Wall -D_LIBRARY -DUSEVEC -DUSEVECREG -DUSEAESNI -funsafe-math-optimizations -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -maes -Wa,-march=generic64+ae
s -Wa,-ahlms=/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/aes-ncbi.aes-ni.pic.o.list /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/ae
s-ncbi.c
In file included from /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/aes-ncbi.c:29:0:
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/ncbi-priv.h:60:33: warning: ‘KAESBlockCipherVecAesNiaes_ncbi_name’ defined but not used [-Wunused-const-v
ariable=]
# define AESBCMEMBER(name) KAESBlockCipherVecAesNi##name
^
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/aes-ncbi.c:2292:19: note: in expansion of macro ‘AESBCMEMBER’
static const char AESBCMEMBER(aes_ncbi_name) [] = CLASS_STRING(AES_CLASS);
^~~~~~~~~~~
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/aes-ncbi.c:95:11: warning: ‘vec_10’ defined but not used [-Wunused-const-variable=]
CipherVec vec_10 = { 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10,
^~~~~~
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/aes-ncbi.c: In function ‘KAESBlockCipherVecAesNiMake’:
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/aes-ncbi.c:2507:34: warning: ‘d’ may be used uninitialized in this function [-Wmaybe-uninitialized]
uint32_t level = 1, a, b, c, d;
^
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/aes-ncbi.c:2507:31: warning: ‘c’ may be used uninitialized in this function [-Wmaybe-uninitialized]
uint32_t level = 1, a, b, c, d;
^
/tmp/cce3Rlg0.s: Assembler messages:
/tmp/cce3Rlg0.s:70: Error: pshufb' is not supported on generic64.aes'
..
..
...
/tmp/cce3Rlg0.s:611: Error: pshufb' is not supported on generic64.aes'
/tmp/cce3Rlg0.s:615: Error: pshufb' is not supported on generic64.aes'
make[8]: *** [/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/libs/krypto/Makefile:194: aes-ncbi.aes-ni.pic.o] Error 1
make[8]: Target '/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/ncbi-out/ncbi-vdb/linux/gcc/x86_64/rel/std' not remade because of errors.
reattempting without AES-NI instructions
Installing libraries to /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/lib64
Installing includes to /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/include
Installing configuration files to /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/etc/ncbi
Updating /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/usr/include/ncbi-vdb
Updating /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/etc/profile.d/ncbi-vdb.[c]sh
Use $NCBI_VDB_LIBDIR in your link commands, e.g.:
ld -L$NCBI_VDB_LIBDIR -lncbi-vdb ...
Please add /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/lib64 to your LD_LIBRARY_PATH, e.g.:
export LD_LIBRARY_PATH=/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/lib64:$LD_LIBRARY_PATH
Use /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/lib64 in your link commands, e.g.:
export NCBI_VDB_LIBDIR=/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install/root/lib64
ld -L$NCBI_VDB_LIBDIR -lncbi-vdb ...
make[3]: Leaving directory '/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install'
make[2]: Leaving directory '/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/install'
make[1]: Target 'by_default' not remade because of errors.
make[1]: Leaving directory '/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test'

[yangming]

another test error:

g++ -c -DNDEBUG -m64 -DLINUX -DUNIX -D_GNU_SOURCE -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPKGNAME=linux64 -D_ARCH_BITS=SIZEOF_POINTER*CHAR_BIT -DLIBPREFIX=lib
-DSHLIBEXT=so -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/override -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/linux -I/home/east
asia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/unix -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vd
b/interfaces -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/cc/gcc/x86_64 -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/cc/gcc -I/hom
e/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/os/linux -I/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/interfaces/os/unix -I/home/eastasia/softhouse/bi
otools/git_svn/ncbi-vdb/interfaces/ext -I/usr/include -I. -MD -o resolvertest.o -O3 -Wno-variadic-macros -fno-strict-aliasing -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -Wal
l /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/resolvertest.cpp -D__mod_name__=test/vfs -D__file_name__=resolvertest -D__file_ext__=cpp
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/resolvertest.cpp: In destructor ‘ResolverFixtureCustomConfig::~ResolverFixtureCustomConfig()’:
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/resolvertest.cpp:196:90: warning: throw will always call terminate() [-Wterminate]
throw logic_error ( "~ResolverFixtureCustomConfig: VFSManagerRelease failed" );
^
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/resolvertest.cpp:196:90: note: in C++11 destructors default to noexcept
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/resolvertest.cpp:199:89: warning: throw will always call terminate() [-Wterminate]
throw logic_error ( "~ResolverFixtureCustomConfig: VResolverRelease failed" );
^
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/resolvertest.cpp:199:89: note: in C++11 destructors default to noexcept
g++ -o /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/ncbi-out/ncbi-vdb/linux/gcc/x86_64/rel/test-bin/test-resolver -DNDEBUG -m64 resolvertest.o -L/home/eastasia/so
fthouse/biotools/git_svn/ncbi-vdb/ncbi-out/ncbi-vdb/linux/gcc/x86_64/rel/lib -L/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/ncbi-out/ncbi-vdb/linux/gcc/x86_64/rel
/ilib -Wl,-Bstatic -lkapp -lktst -lncbi-vdb -Wl,-Bdynamic -ldl -lpthread -lm
make[3]: Leaving directory '/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/ncbi-out/ncbi-vdb/linux/gcc/x86_64/rel/obj/test/vfs'
++++++++++++++++++++++++++++++++++++++++++++++++++++++
Run /home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/ncbi-out/ncbi-vdb/linux/gcc/x86_64/rel/test-bin/redirect-rejected-names-cgi-http-to-https
Running 1 test case...
/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs/redirect-rejected-names-cgi-http-to-https.cpp:126: fatal error in "TEST.HTTP": Unexpected return code from V
ResolverQuery ( resolver, 0, query, 0, & remote, 0 ): RC(rcVFS,rcTree,rcResolving,rcPath,rcNotFound)

*** 1 failure detected in test suite "VResolverTestSuite"
make[2]: *** [/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/build/Makefile.env:345: runtests] Error 3
make[2]: Leaving directory '/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/test/vfs'
make[1]: *** [/home/eastasia/softhouse/biotools/git_svn/ncbi-vdb/build/Makefile.targets:90: vfs_runtests] Error 2

[kwrodarmer]

Can you please tell us the version(s) of compiler you are running?

[klymenko]

redirect-rejected-names-cgi-http-to-https: this test required you to have HTTP/HTTPS access to NCBI.

[yangming]

$ gcc --version
gcc (GCC) 6.3.1 20170306
Copyright (C) 2016 Free Software Foundation, Inc.

this is the compiler i used.

for https, i can access the ncbi webpage through https. for the test, were we required to get the ca file? if true, where did i get it?

[jbaksta]

Also seeing this on RHEL 6.8 and non-vendor GCC 6.2.0 compiler.

[kwrodarmer]

Thank you - we'll try to reproduce.

[kwrodarmer]

We were unable to duplicate the error. We can try again.

[dcjones]

This (also documented in #6) occurs because assembler flags are incorrectly set here and in the line below it.

ncbi-vdb/libs/krypto/Makefile

Line 138 in 09046b4

_CC_AES_NI = -funsafe-math-optimizations -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -maes -Wa,-march=generic64+aes $(CC_LISTING)

A bunch of instruction sets are enabled for the compiler, then not enabled for the assembler, which can cause these errors. Probably the whole -Wa,-march=... argument should be deleted to avoid this. It doesn't make any sense to set the assembler to a more restricted instruction set than what the compiler is using.

[bioinfornatics]

Thanks @dcjones
I solved with your comment

sed -i 's/-Wa,-march=generic64+[[:alnum:]]*//g' libs/krypto/Makefile