TFLite export issue from yolov3.pt weights (works for yolov3-spp.pt)

[corehalt]

Search before asking

• I have searched the YOLOv3 issues and found no similar bug report.

YOLOv3 Component

Export

Bug

python export.py --weights yolov3.pt --include saved_model pb tflite --int8 --img 416
export: data=data/coco128.yaml, weights=yolov3.pt, imgsz=[416], batch_size=1, device=cpu, half=False, inplace=False, train=False, optimize=False, int8=True, dynamic=False, simplify=False, opset=13, topk_per_class=100, topk_all=100, iou_thres=0.45, conf_thres=0.25, include=['saved_model', 'pb', 'tflite']
YOLOv3 🚀 v9.6.0-6-g0f80f2f torch 1.10.1+cu102 CPU

Fusing layers... 
Model Summary: 261 layers, 61922845 parameters, 0 gradients, 156.1 GFLOPs

PyTorch: starting from yolov3.pt (124.3 MB)
2022-01-17 16:05:01.871223: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/corehalt/src/yolov3_test/venv/lib/python3.8/site-packages/cv2/../../lib64:
2022-01-17 16:05:01.871262: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

TensorFlow saved_model: starting export with tensorflow 2.4.1...

                 from  n    params  module                                  arguments                     
  0                -1  1       928  models.common.Conv                      [3, 32, 3, 1]                 
2022-01-17 16:05:03.584306: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2022-01-17 16:05:03.584784: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/corehalt/src/yolov3_test/venv/lib/python3.8/site-packages/cv2/../../lib64:
2022-01-17 16:05:03.584829: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2022-01-17 16:05:03.584867: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (pop-os): /proc/driver/nvidia/version does not exist
2022-01-17 16:05:03.585266: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-01-17 16:05:03.585612: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
  1                -1  1     18560  models.common.Conv                      [32, 64, 3, 2]                
  2                -1  1     20672  models.common.Bottleneck                [64, 64]                      
  3                -1  1     73984  models.common.Conv                      [64, 128, 3, 2]               
  4                -1  2    164608  models.common.Bottleneck                [128, 128]                    
  5                -1  1    295424  models.common.Conv                      [128, 256, 3, 2]              
  6                -1  8   2627584  models.common.Bottleneck                [256, 256]                    
  7                -1  1   1180672  models.common.Conv                      [256, 512, 3, 2]              
  8                -1  8  10498048  models.common.Bottleneck                [512, 512]                    
  9                -1  1   4720640  models.common.Conv                      [512, 1024, 3, 2]             
 10                -1  4  20983808  models.common.Bottleneck                [1024, 1024]                  
 11                -1  1   5245952  models.common.Bottleneck                [1024, 1024, False]           

TensorFlow saved_model: export failure: Convolution with multiple kernels are not allowed.

TensorFlow GraphDef: starting export with tensorflow 2.4.1...

TensorFlow GraphDef: export failure: 'NoneType' object has no attribute 'inputs'

TensorFlow Lite: starting export with tensorflow 2.4.1...

TensorFlow Lite: export failure: 'NoneType' object has no attribute 'call'

Export complete (5.98s)
Results saved to /home/corehalt/src/yolov3_test/yolov3
Visualize with https://netron.app

Environment

• YOLO: YOLOv3 🚀 v9.6.0-6-g0f80f2f torch 1.10.1+cu102 CPU
• OS: Ubuntu 20.04 (Pop OS)
• Python: Python 3.8.1

Minimal Reproducible Example

python export.py --weights yolov3.pt --include saved_model pb tflite --int8 --img 416

Additional

I'm trying to export the yolov3 model to TFLite.
I installed the requirements in a clean virtualenv with Python 3.8.10 (uncommenting the tflite export section in the file) and run the export command:

# works with `yolov3-spp.pt` (automatically downloaded previously by detect.py)
python export.py --weights yolov3-spp.pt --include saved_model pb tflite --int8 --img 416

but, on the other hand, it fails to export the standard yolov3.pt model. It fails with the specified error:

# Fails with `yolov3.pt` (automatically downloaded previously by detect.py)
python export.py --weights yolov3.pt --include saved_model pb tflite --int8 --img 416

I also tried to to use different versions of TensorFlow (2.7.0, 2.6.1, 2.4.1) but doesn't work as well.

Thank you.

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[github-actions]

👋 Hello @corehalt, thank you for your interest in YOLOv3 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov3
$ cd yolov3
$ pip install -r requirements.txt

Environments

YOLOv3 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv3 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv3 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@corehalt yes I'm able to reproduce the same. Not sure what the issue might be. You could try exporting via the YOLOv5 repo, which is more up to date, though the repo doesn't seem to be the problem since YOLOv3-SPP exports correctly.

[glenn-jocher]

@corehalt unfortunately our support levels for the YOLOv3 repo are minimal as we support and grow YOLOv5, so I'll put a TODO on this, but I can't give you an ETA for when it might be solved.

[corehalt]

@glenn-jocher just for the record, same issue also happens for yolov3-tiny.pt.

[glenn-jocher]

@corehalt hmm interesting. We probably just need to retrain all the YOLOv3 models with the latest YOLOv5 master, i.e. just use the yolov3 yamls with the YOLOv5 repo and retrain, and then these should be exportable.

When we released these models TFLite export was not fully supported, and it's still not part of our CI checks (though it should be).

[glenn-jocher]

@corehalt all https://github.com/ultralytics/yolov3 models are now retrained but original issue is not resolved with these new models.

New models are attached as updated release assets at https://github.com/ultralytics/yolov3/releases/tag/v9.6.0

[corehalt]

@glenn-jocher Thank you for retraining the models.
An update on this is that even using the new yolov3 retrained model under yolov5 repository I'm still getting the same issue:

TensorFlow Lite: export failure: 'NoneType' object has no attribute 'call'

[glenn-jocher]

@corehalt yeah sorry. Can you try debugging by removing the TFLite export try: except and tracing the source of the error?

[corehalt]

@glenn-jocher I think I found the issue, now I can export correctly using yolov3.yaml
Seems that here https://github.com/ultralytics/yolov3/blob/master/models/yolov3.yaml#L31 there is a typo:

# YOLOv3 head
head:
  [[-1, 1, Bottleneck, [1024, False]],
   [-1, 1, Conv, [512, [1, 1]]],   <-------
   [-1, 1, Conv, [1024, 3, 1]],

to be able to export later in TF it should be changed to:

# YOLOv3 head
head:
  [[-1, 1, Bottleneck, [1024, False]],
   [-1, 1, Conv, [512, 1, 1]],   <------ no brackets
   [-1, 1, Conv, [1024, 3, 1]],

The problem is that any training that starts from the pre-trained yolov3.pt model will have this issue.
Only a train from the scratch will allow to export later to TF.

Is it possible for you to retrain the model with the updated yaml?

Thank you

[corehalt]

Ah, the typo also appears in the yaml file configuration of the YoloV5 repo:
https://github.com/ultralytics/yolov5/blob/master/models/hub/yolov3.yaml#L31

[glenn-jocher]

@corehalt oh awesome, nice work! Yes I can update and retrain.