Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
282caa7
Upgrade TF dependency to v2.5-rc0.
AdamHillier Apr 10, 2021
3db553d
Allow up to 10 Mb of logging.
AdamHillier Apr 11, 2021
4b05ed4
Use TF definition of manylinux toolchain.
AdamHillier Apr 11, 2021
83e6b75
Use embedded linux toolchains, plus some review suggestions.
AdamHillier Apr 12, 2021
86931d5
Fix typo in .bazelrc
AdamHillier Apr 12, 2021
3cc7ecd
Partly fix MLIR issues.
AdamHillier Apr 12, 2021
4648903
Disable framework_shared_object for all LCE builds and tests.
AdamHillier Apr 13, 2021
7cdc087
Fix MLIR errors in prepare-tf.
AdamHillier Apr 13, 2021
ad9b9ed
Remove TF_SYSTEM_LIBS and fix typo.
AdamHillier Apr 13, 2021
ebb2ba4
Fix end2end test errors.
AdamHillier Apr 13, 2021
6aac46f
:arrow_up: [email protected] (#632)
lgeiger Apr 13, 2021
cf72fdb
Add Python 3.9 to tests/releases.
AdamHillier Apr 14, 2021
b10e6d0
Merge remote-tracking branch 'upstream/master' into tf-2.5
AdamHillier Apr 14, 2021
ef87a22
Split TF options out of .bazelrc.
AdamHillier Apr 14, 2021
1ac22b8
Apply full static linking only to the benchmarker target.
AdamHillier Apr 14, 2021
8c146a6
Temporarily re-enable GCP/AWS on Windows.
AdamHillier Apr 14, 2021
7c6599a
Hopefully fix Windows config 🤞
AdamHillier Apr 14, 2021
99a780c
Use nightly-custom-op-ubuntu16 Docker image for ManyLinux builds.
AdamHillier Apr 14, 2021
4cf2ad3
Don't build 3.9 wheels on Linux, bump Windows pagefile size.
AdamHillier Apr 15, 2021
3673a7f
Update manylinux toolchain to match upstream TF version
lgeiger Apr 26, 2021
ebd4b06
Fix Python 3.9 builds on Linux
lgeiger Apr 26, 2021
8839bc2
Add bazel memory saving flags to Linux builds
lgeiger Apr 26, 2021
0e75d76
Update signature_defs tensor indices after quantization (#635)
lgeiger Apr 27, 2021
04dce42
Remove TensorFlow TD includes from LCE ops
lgeiger Apr 27, 2021
9d56ea5
Fix CI
lgeiger Apr 27, 2021
842cd6f
:arrow_up: [email protected]
lgeiger Apr 28, 2021
3222314
:arrow_up: TF version and disable GCP/AWS on Windows.
AdamHillier May 3, 2021
634d95c
Remove conflicting Windows configs.
AdamHillier May 4, 2021
9922add
Build with `fastbuild` on Windows.
AdamHillier May 4, 2021
e5701ba
:arrow_up: v2.5.0 and add patch for tensorflow/tensorflow#48546.
AdamHillier May 19, 2021
09fc53f
Disable fail-fast for release matrix jobs.
AdamHillier May 19, 2021
a6a0f50
Small changes.
AdamHillier May 19, 2021
a7aa08e
Don't statically link benchmark binary on Android.
AdamHillier May 19, 2021
b5ec1b1
Update passes to match TF 2.5 (#643)
lgeiger May 19, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
151 changes: 39 additions & 112 deletions .bazelrc
Original file line number Diff line number Diff line change
@@ -1,129 +1,56 @@
build --spawn_strategy=standalone
build --strategy=Genrule=standalone
build -c opt
# Import TensorFlow's configuration first.
try-import %workspace%/.tensorflow.bazelrc

# Disable visibility checks (works around some private deps in TensorFlow).
# Prevent invalid caching if input files are modified during a build.
build --experimental_guard_against_concurrent_changes

# Allow up to 10 Mb of logging
build --experimental_ui_max_stdouterr_bytes=10485760

# Disable visibility checks (works around some private deps in TensorFlow)
build --nocheck_visibility

# For compatibility with TensorFlow config
common --experimental_repo_remote_exec
# Disable framework_shared_object for all LCE builds and tests.
build --config=monolithic

# Make sure tests are quick and -DNDEBUG is *not* set
test --compilation_mode=fastbuild

# Enable Ruy
build --copt=-DTFLITE_WITH_RUY

# Disable XLA
build --define=with_xla_support=false

# Disable MKL
build --define=enable_mkl=false --define=build_with_mkl=false --define=tensorflow_mkldnn_contraction_kernel=0

# By default, build TF in C++ 14 mode.
build:android --cxxopt=-std=c++14
build:android --host_cxxopt=-std=c++14
build:linux --cxxopt=-std=c++14
build:linux --host_cxxopt=-std=c++14
build:macos --cxxopt=-std=c++14
build:macos --host_cxxopt=-std=c++14
build:windows --cxxopt=/std:c++14
build:windows --host_cxxopt=/std:c++14

# These can be activated using --config=rpi3 and --config=aarch64
build:rpi3 --crosstool_top=@local_config_arm_compiler//:toolchain
build:rpi3 --cpu=armeabi
build:rpi3 --copt=-march=armv7-a --copt=-mfpu=neon-vfpv4 --copt=-std=gnu++11 --copt=-DS_IREAD=S_IRUSR --copt=-DS_IWRITE=S_IWUSR --copt=-fno-tree-pre --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 --define=raspberry_pi_with_neon=true --define=framework_shared_object=false --copt=-funsafe-math-optimizations --copt=-ftree-vectorize --copt=-fomit-frame-pointer --verbose_failures

build:aarch64 --crosstool_top=@local_config_arm_compiler//:toolchain
build:aarch64 --cpu=aarch64
build:aarch64 --copt=-march=armv8-a --copt=-std=gnu++11 --copt=-DS_IREAD=S_IRUSR --copt=-DS_IWRITE=S_IWUSR --copt=-fno-tree-pre --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 --define=framework_shared_object=false --copt=-funsafe-math-optimizations --copt=-ftree-vectorize --copt=-fomit-frame-pointer --verbose_failures

# Options to build TensorFlow 1.x or 2.x.
build:v1 --define=tf_api_version=1
build:v2 --define=tf_api_version=2
test:v1 --action_env=TF2_BEHAVIOR=0
test:v2 --action_env=TF2_BEHAVIOR=1

build --config=v2
test --config=v2 --compilation_mode=fastbuild

build --define=grpc_no_ares=true

# Options to disable default on features
build:noaws --define=no_aws_support=true
build:nogcp --define=no_gcp_support=true
build:nohdfs --define=no_hdfs_support=true
build:nonccl --define=no_nccl_support=true

build:linux --config=noaws --config=nogcp --config=nohdfs --config=nonccl
build:macos --config=noaws --config=nogcp --config=nohdfs --config=nonccl

# Tensorflow uses M_* math constants that only get defined by MSVC headers if
# _USE_MATH_DEFINES is defined.
build:windows --copt=/D_USE_MATH_DEFINES
build:windows --host_copt=/D_USE_MATH_DEFINES

# Config to use a mostly-static build and disable modular op registration
# support (this will revert to loading TensorFlow with RTLD_GLOBAL in Python).
# By default, TensorFlow will build with a dependence on
# //tensorflow:libtensorflow_framework.so.
build:monolithic --define framework_shared_object=false

# On windows, we still link everything into a single DLL.
build:windows --config=monolithic

# Make sure to include as little of windows.h as possible
build:windows --copt=-DWIN32_LEAN_AND_MEAN
build:windows --host_copt=-DWIN32_LEAN_AND_MEAN
build:windows --copt=-DNOGDI
build:windows --host_copt=-DNOGDI

# MSVC (Windows): Standards-conformant preprocessor mode
# See https://docs.microsoft.com/en-us/cpp/build/reference/zc-preprocessor
build:windows --copt=/Zc:preprocessor
build:windows --host_copt=/Zc:preprocessor

# Misc build options we need for windows.
build:windows --linkopt=/DEBUG
build:windows --host_linkopt=/DEBUG
build:windows --linkopt=/OPT:REF
build:windows --host_linkopt=/OPT:REF
build:windows --linkopt=/OPT:ICF
build:windows --host_linkopt=/OPT:ICF
build:windows --copt=/d2ReducedOptimizeHugeFunctions
build:windows --host_copt=/d2ReducedOptimizeHugeFunctions
# Config for a 32-bit Raspberry Pi - can be activated using --config=rpi3
build:rpi3 --config=elinux_armhf
build:rpi3 --copt=-march=armv7-a --copt=-mfpu=neon-vfpv4 --copt=-std=gnu++11 --copt=-DS_IREAD=S_IRUSR --copt=-DS_IWRITE=S_IWUSR --copt=-fno-tree-pre --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 --define=raspberry_pi_with_neon=true --copt=-funsafe-math-optimizations --copt=-ftree-vectorize --copt=-fomit-frame-pointer --verbose_failures

# Config for 64-bit ARM - can be activated using --config=aarch64
build:aarch64 --config=elinux_aarch64
build:aarch64 --copt=-march=armv8-a --copt=-std=gnu++11 --copt=-DS_IREAD=S_IRUSR --copt=-DS_IWRITE=S_IWUSR --copt=-fno-tree-pre --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 --copt=-funsafe-math-optimizations --copt=-ftree-vectorize --copt=-fomit-frame-pointer --verbose_failures

# Disable unnecessary features.
build:linux --config=nohdfs --config=nonccl --config=noaws --config=nogcp
build:macos --config=nohdfs --config=nonccl --config=noaws --config=nogcp
build:windows --config=noaws --config=nogcp

# Extra build options we need for windows.
build:windows --experimental_strict_action_env=true

# Verbose failure logs when something goes wrong
build:windows --verbose_failures
# Disable certain warnings that come from TF code and pollute logs
build:android --copt=-Wno-deprecated-declarations
build:linux --copt=-Wno-deprecated-declarations
build:linux --host_copt=-Wno-deprecated-declarations
build:macos --copt=-Wno-deprecated-declarations

# Suppress C++ compiler warnings, otherwise build logs become 10s of MBs.
# Windows generates *a lot* of warnings; disable them all.
build:windows --copt=/W0

# On windows, we never cross compile
build:windows --distinct_host_configuration=false

build --define=use_fast_cpp_protos=true
build --define=allow_oversize_protos=true

# Enable using platform specific build settings, except when cross-compiling for
# mobile platforms.
build --enable_platform_specific_config
build:android --noenable_platform_specific_config

# Android configs. Bazel needs to have --cpu and --fat_apk_cpu both set to the
# target CPU to build transient dependencies correctly. See
# https://docs.bazel.build/versions/master/user-manual.html#flag--fat_apk_cpu
build:android --crosstool_top=//external:android/crosstool
build:android --host_crosstool_top=@bazel_tools//tools/cpp:toolchain
build:android_arm --config=android
build:android_arm --cpu=armeabi-v7a
build:android_arm --fat_apk_cpu=armeabi-v7a
build:android_arm64 --config=android
build:android_arm64 --cpu=arm64-v8a
build:android_arm64 --fat_apk_cpu=arm64-v8a
build:android_x86 --config=android
build:android_x86 --cpu=x86
build:android_x86 --fat_apk_cpu=x86
build:android_x86_64 --config=android
build:android_x86_64 --cpu=x86_64
build:android_x86_64 --fat_apk_cpu=x86_64

# Default options should come above this line
# Config-specific options should come above this line.

# Options from ./configure
try-import %workspace%/.lce_configure.bazelrc
Expand Down
2 changes: 1 addition & 1 deletion .bazelversion
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.1.0
3.7.2
6 changes: 4 additions & 2 deletions .github/tools/release_linux.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,14 @@ ln -sf /usr/include/x86_64-linux-gnu/python3.7m /dt7/usr/include/x86_64-linux-gn
ln -sf /usr/include/x86_64-linux-gnu/python3.7m /dt8/usr/include/x86_64-linux-gnu/python3.7m || true
ln -sf /usr/include/x86_64-linux-gnu/python3.8 /dt7/usr/include/x86_64-linux-gnu/python3.8 || true
ln -sf /usr/include/x86_64-linux-gnu/python3.8 /dt8/usr/include/x86_64-linux-gnu/python3.8 || true
ln -sf /usr/include/x86_64-linux-gnu/python3.9 /dt7/usr/include/x86_64-linux-gnu/python3.9 || true
ln -sf /usr/include/x86_64-linux-gnu/python3.9 /dt8/usr/include/x86_64-linux-gnu/python3.9 || true

curl https://bootstrap.pypa.io/get-pip.py | python
python --version
python -m pip --version

curl -L https://github.com/bazelbuild/bazelisk/releases/download/v1.7.4/bazelisk-linux-amd64 > /usr/bin/bazelisk
curl -L https://github.com/bazelbuild/bazelisk/releases/download/v1.8.0/bazelisk-linux-amd64 > /usr/bin/bazelisk
chmod +x /usr/bin/bazelisk

python -m pip install numpy six --no-cache-dir
Expand All @@ -33,7 +35,7 @@ export MANYLINUX2010=1
./configure.py

# Build
bazelisk build :build_pip_pkg --copt=-fvisibility=hidden --copt=-mavx --distinct_host_configuration=false
bazelisk build :build_pip_pkg --copt=-fvisibility=hidden --copt=-mavx --distinct_host_configuration=false --discard_analysis_cache --notrack_incremental_state --nokeep_state_after_build --nouse_action_cache --local_ram_resources=4096

# Package Whl
bazel-bin/build_pip_pkg artifacts
Expand Down
17 changes: 10 additions & 7 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: 3.8
python-version: 3.9
- uses: actions/[email protected]
id: cache
with:
Expand All @@ -39,11 +39,11 @@ jobs:
- run: mkdir benchmark-binaries
- name: Build Benchmark utility for AArch64
run: |
bazelisk build //larq_compute_engine/tflite/benchmark:lce_benchmark_model --config=aarch64 -c opt --copt=-O3 --features=fully_static_link
bazelisk build //larq_compute_engine/tflite/benchmark:lce_benchmark_model --config=aarch64 -c opt --copt=-O3
cp bazel-bin/larq_compute_engine/tflite/benchmark/lce_benchmark_model benchmark-binaries/lce_benchmark_model_aarch64
- name: Build Benchmark utility for AArch32
run: |
bazelisk build //larq_compute_engine/tflite/benchmark:lce_benchmark_model --config=rpi3 -c opt --copt=-O3 --features=fully_static_link
bazelisk build //larq_compute_engine/tflite/benchmark:lce_benchmark_model --config=rpi3 -c opt --copt=-O3
cp bazel-bin/larq_compute_engine/tflite/benchmark/lce_benchmark_model benchmark-binaries/lce_benchmark_model_aarch32
- name: Build Benchmark utility for Android
run: |
Expand Down Expand Up @@ -133,7 +133,8 @@ jobs:
runs-on: macos-latest
strategy:
matrix:
python-version: [3.6, 3.7, 3.8]
python-version: [3.6, 3.7, 3.8, 3.9]
fail-fast: false
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
Expand Down Expand Up @@ -165,7 +166,8 @@ jobs:
runs-on: ubuntu-18.04
strategy:
matrix:
python-version: [3.6, 3.7, 3.8]
python-version: [3.6, 3.7, 3.8, 3.9]
fail-fast: false
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
Expand Down Expand Up @@ -197,12 +199,13 @@ jobs:
runs-on: windows-latest
strategy:
matrix:
python-version: [3.6, 3.7, 3.8]
python-version: [3.6, 3.7, 3.8, 3.9]
fail-fast: false
steps:
- name: Configure Pagefile
uses: al-cheb/[email protected]
with:
minimum-size: 4GB
minimum-size: 8GB
- name: Pagefile size
run: |
(Get-CimInstance Win32_PageFileUsage).AllocatedBaseSize
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/unittests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
sudo apt-get install -y --no-install-recommends qemu-user
- uses: actions/setup-python@v2
with:
python-version: 3.8
python-version: 3.9
- name: Configure Bazel
run: ./configure.py
shell: bash
Expand Down Expand Up @@ -72,7 +72,7 @@ jobs:
export_default_credentials: true
- uses: actions/setup-python@v2
with:
python-version: 3.8
python-version: 3.9
- name: Configure Bazel
run: |
./configure.py
Expand All @@ -88,7 +88,7 @@ jobs:
if: github.ref != 'refs/heads/master'
shell: bash
- name: Install pip dependencies
run: pip install tensorflow-cpu~=2.4 larq~=0.11 larq_zoo~=2.0 pytest tensorflow_datasets~=4.2 flatbuffers tqdm --no-cache-dir
run: pip install tensorflow-cpu~=2.5.0 larq~=0.11 larq_zoo~=2.0 pytest tensorflow_datasets~=4.2 flatbuffers tqdm --no-cache-dir
- name: Run Interpreter test
run: bazelisk test larq_compute_engine/tflite/tests:interpreter_test --test_output=all
- name: Run FileCheck tests
Expand All @@ -100,7 +100,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
tf-version: [1.14.0, 1.15.5, 2.0.4, 2.1.3, 2.2.2, 2.3.2, 2.4.1]
tf-version: [1.14.0, 1.15.5, 2.0.4, 2.1.3, 2.2.2, 2.3.2, 2.4.1, 2.5.0]
if: "!contains(github.event.head_commit.message, 'ci-skip')"
steps:
- uses: actions/checkout@v2
Expand All @@ -122,7 +122,7 @@ jobs:
fetch-depth: 0
- uses: actions/setup-python@v2
with:
python-version: 3.8
python-version: 3.9
- uses: actions/[email protected]
id: cache
with:
Expand Down
Loading