Adding support for Hailo

[aallan]

To setup a Hailo environment with TensorFlow we'd need to clone the Hailo examples repository, set up the Hailo environment — this creates the virtual environment, and exports the TAPPAS post processing directory. Then,

pip install -r Repos/hailo-rpi5-examples/requirements.txt

pip install numpy==1.26.2 
pip install keras_applications==1.0.8 --no-deps 
pip install keras_preprocessing==1.1.2 --no-deps 
pip install h5py==3.10.0 
pip install pybind11==2.9.2 
pip install packaging 
pip install protobuf==3.20.3 
pip install six wheel mock gdown 
pip install opencv-python
TFVER=2.15.0.post1
PYVER=311
ARCH=`python -c 'import platform; print(platform.machine())'`
pip install --no-cache-dir https://github.com/PINTO0309/Tensorflow-bin/releases/download/v${TFVER}/tensorflow-${TFVER}-cp${PYVER}-none-linux_${ARCH}.whl

Then we need to write a script to run the Dataflow Compiler to convert our TensorFlow models to HEF format, so it can be used by the Hailo module.

[aallan]

The Dataflow Compiler can't be installed on Raspberry Pi

[aallan]

May now be coming "soon" ..?

[aallan]

The DFC is now available!

We’ve heard your feedback, and we’re excited to announce that access to the Dataflow Compiler (DFC) is now available! With that, we would like to help you set the right approach towards using the DFC.

Bring Your Own Data (BYOD) - The most recommended way is using the retraining Docker containers from the Hailo Model Zoo. This will allow you to use some of the most popular networks, (re)trained on your own custom data (see this link for an end-to-end example 46 in our GitHub repo).

Bring Your Own Model (BYOM) – This method requires a deeper understanding of the model and the conversion flow from ONNX or TensorFlow Lite (TFLite) to Hailo’s HEF format. We suggest using the DFC user guide and tutorials. This approach may present more challenges compared to using the retraining Docker containers from the Model Zoo, but for some, the ride is as enjoyable as the destination 😊 (see this link for DFC tutorials 32).

Large models (e.g., LLMs, Whisper, Stable-Diffusion, etc.) include some of the SoTA models and are naturally very attractive. We’re actively working on enabling the porting of these models to Hailo, but at this point in time, we cannot yet execute these on the Hailo-8L.

Please bear with us as we embark on this adventure of releasing the DFC. We are committed to giving the best experience that we can deliver, and we believe this is another step in enabling exciting projects 😊

More at https://www.raspberrypi.com/news/raspberry-pi-ai-kit-update-dataflow-compiler-now-available/.

[aallan]

Should look at the Seeed Studio YOLO benchmark code and see if that's helpful for integration. I think the real sticking point here is getting the benchmark model translated to HEF format, the DFC doesn't look straighforward.