Instrumentation cleanup when operation was removed

[AlexandreEichenberger]

When some later optimizations remove an op that is instrumented, we still get the BEFORE/AFTER instrumentation back to back without anything in between.

That makes it harder to detect if a given optimization removes operations or not. For example, when -enable-compiler-stick-unstick succeeds or not in removing unneeded stick/unstick.

This PR adds a pass just before lowering instrumentation and detects if there are 2 consecutive instrumentation call (one BEFORE and one AFTER tags) that do not also initialize the instrumentation. When the is the case, then these two instrumentation operations are removed.

Now we can directly see if an optimization add/remove operations using the make-report.py and get more informative profiling information as the number of calls more precisely reflect the number of actual operations executed.

Measurement without compiler gen stick/unstick (1 stick/unstick only):

root@5063a313c8ac:/workdir/amodel/ccfd# make-report.py -w 2 -uus -r run-perf-zdnn.log

Statistics start all ops ordered_by time, tot_time,  1316.0000000
  zhigh.LSTM, 2, 550.4166667, 1100.8333333, 83.6%
  zhigh.Stick, 1, 159.0000000, 159.0000000, 12.1%
  zhigh.MatMul, 1, 28.0000000, 28.0000000, 2.1%
  zhigh.Add, 1, 9.0000000, 9.0000000, 0.7%
  zhigh.Sigmoid, 1, 9.0000000, 9.0000000, 0.7%
  zhigh.StickifiedConstant, 14, 0.3809524, 5.3333333, 0.4%
  zhigh.Unstick, 1, 2.0000000, 2.0000000, 0.2%
  onnx.Constant, 1, 1.0000000, 1.0000000, 0.1%
Statistics end all ops ordered_by time, tot_time,  1316.0000000

Measurement with compiler gen stick/unstick (3 stick/unstick):

root@5063a313c8ac:/workdir/amodel/ccfd# make-report.py -w 2 -uus -r run-perf-csu.log
Statistics start all ops ordered_by time, tot_time,  1351.0000000
  zhigh.LSTM, 2, 553.4166667, 1106.8333333, 81.9%
  zhigh.Unstick, 3, 41.4444444, 124.3333333, 9.2%
  zhigh.Stick, 3, 21.2222222, 63.6666667, 4.7%
  zhigh.MatMul, 1, 28.0000000, 28.0000000, 2.1%
  zhigh.Sigmoid, 1, 9.3333333, 9.3333333, 0.7%
  zhigh.Add, 1, 8.8333333, 8.8333333, 0.7%
  zhigh.StickifiedConstant, 14, 0.4047619, 5.6666667, 0.4%
  onnx.Constant, 1, 0.6666667, 0.6666667, 0.0%
  onnx.Squeeze, 2, 0.1666667, 0.3333333, 0.0%
Statistics end all ops ordered_by time, tot_time,  1351.0000000

[chentong319]

LGTM!

[jenkins-droid]

Jenkins Linux s390x Build #16227 [push] Instrumentation cleanup ... started at 14:25

[jenkins-droid]

Jenkins Linux ppc64le Build #15254 [push] Instrumentation cleanup ... started at 14:45

[jenkins-droid]

Jenkins Linux amd64 Build #16225 [push] Instrumentation cleanup ... started at 13:26

[jenkins-droid]

Jenkins Linux amd64 Build #16225 [push] Instrumentation cleanup ... passed after 1 hr 17 min

[jenkins-droid]

Jenkins Linux s390x Build #16227 [push] Instrumentation cleanup ... passed after 1 hr 27 min

[jenkins-droid]

Jenkins Linux ppc64le Build #15254 [push] Instrumentation cleanup ... passed after 2 hr 32 min