[Audio] Microphone Capture - Allow setting smaller chunk size for low latency

[virajkarandikar]

• I have searched to see if a similar issue already exists.

By default the streaming mic capture uses buffer/chunk size of 1 second. This adds a long latency in real time applications. Can the chunk size be made configurable/smaller?

Is your feature request related to a problem? Please describe.
Large buffer increases audio latency and makes application sluggish to use.

Describe the solution you'd like
Provide a parameter to configure chunk size when using streaming mic capture

Additional context
Add any other context or screenshots about the feature request here.

[abidlabs]

Hi @virajkarandikar can you provide sample code we can use to look at the issue?

[virajkarandikar]

Code is simple.

with gr.Blocks() as demo:
        audio = gr.Audio(streaming=True)

        def process_audio(audio):
            rate, data = audio
            print(f"rate: {rate}, samples: {len(data)}")

        audio.stream(process_audio, [audio], None)

Below is the log I get on console.

rate: 48000, samples: 24000
rate: 48000, samples: 48000
rate: 48000, samples: 24000
rate: 48000, samples: 24000
rate: 48000, samples: 24000
rate: 48000, samples: 24000

Log indicates - sample rate is 48000, channels is 1, chunk size varies between 24000 (0.5 sec) and 48000 (1 sec). This adds significant latency.

Also the uncompressed audio data at 48000Hz is streamed from the client to application and it adds some amount of network latency. My case model expects 16000 sample rate. So if I can specify sample rate for mic capture, it will reduce the amount of data transfer by 1/3rd. But for that I have filed another issue here #5848.

[qianhuiliu]

Hello, have you figured out how to do it? I have the same question.

[virajkarandikar]

Any update here?

[gaborvecsei]

I am also interested in this

[virajkarandikar]

Ping...

[abidlabs]

This is on our radar, but maybe will take a few weeks for us to get to as we have a lot of other issues we're tackling as well. We are happy to review any PRs if you'd like to contribute this fix.

cc @aliabid94

[mcorroyer]

I have the same issue, has it progressed?

[adirajagopal]

This is on our radar, but maybe will take a few weeks for us to get to as we have a lot of other issues we're tackling as well. We are happy to review any PRs if you'd like to contribute this fix.

cc @aliabid94

Hi, is there a specific part of the code base you could point to to suggest how we can reduce the chunk size of the stream? This would help with guiding the PR. Thanks!

[JohanWork]

@abidlabs could you point out where the chunk size is set? Happy to contribute

[abidlabs]

Actually, we've implemented this already in our 5.0-dev branch! Let me point you to the PR where you can install and try it out: #8941

Here's a simple transcription demo where you can set your own stream_every param: #8941 (comment)