How to run online model correctly?

[aczh]

Running the online model using the instructions in the README leads to warped/staticky results.

Steps I took:
Followed the README steps.
Made small modifications to get the shell script to work:

1. I had to modify the requirements.txt slightly. Pulled requirements from this fork: https://github.com/y-otn/CleanMel
2. I had to also downgrade the rich toolbar from 14.1.0 to avoid this bug: [BUG] IndexError: pop from empty list with Rich progress bar in 14.1.0 Textualize/rich#3809

Ran the script using bash inference.sh 0, online S map huggingface

The output is warped, and does not sound good. Example: https://drive.google.com/file/d/1v88Gx2ngGNmuN_NSSInpNKZdXw8P7DGD/view?usp=sharing

The offline models work as expected.

[aczh]

Based on some of the code - I guessing I shouldn't be passing in full files to the online model, and should instead be passing in audio chunks?

[deependujha]

Hi @aczh

I've created PR to fix rich progress bar issue in pytorch-lightning , but ideally, this PR should be merged, as it’s a cleaner and more robust solution.

Thanks,
Deependu

[SaoYear]

Hi, thanks for the interest on our work. The processed audio sounds incorrect. The online model should be able to process the audio streams, there is no need to split the audio to chunks (The performance could decay as the increase of audio length, see OnlineSpatialNet).

I will test the online model and come to this issue later.