Reduce number of simultaneous creature walk sounds. Battlefield

[Districh-ru]

Currently in master build on high battle speeds when creature is moving the walk sound is played for move to every next cell. It results in many simultaneous move sounds playback.

This PR makes engine to play only "about" one sound at a time. "About" is because on low battle speeds we need to avoid possible sound playback delays in case the sound has ended at the beginning of the next turn and we have to listen silence until the next one.

The algorithm: we know the theoretical time for cell to cell move (from game assets) and we know the move sound duration time. So we can calculate how many cells to wait until the next sound playback.
But on high speeds the real move speed may be less than theoretical due to CPU limit or display 'v-sync' limit. So for this case (if creature movement FPS is more than 50 Hz) we start the next sound after a fact that a previous one has ended. A possible pause after the sound end and the next cell step will be not noticeable as the move speed is very high.

[Branikolog]

Hi, @Districh-ru !

2023-06-18.13-16-08.mp4

I hope my ears don't lie to me. It seems that the last sound played in a sequence is cut. It is noticeable for walking creatures, especially making 1 single step. Is this behaviour intended? While making two or more steps sounds seem to be played fully (or closer to that). Curious thing is that flying sound of Dragons and Phoenixes is not cut.
Also, I can hear light clicks, while Air Elementals moving. I believe this is step sound being cut in the middle of sequence...

[Districh-ru]

Hi, @Districh-ru !
I hope my ears don't lie to me. It seems that the last sound played in a sequence is cut. It is noticeable for walking creatures, especially making 1 single step. Is this behaviour intended? While making two or more steps sounds seem to be played fully (or closer to that). Curious thing is that flying sound of Dragons and Phoenixes is not cut. Also, I can hear light clicks, while Air Elementals moving. I believe this is step sound being cut in the middle of sequence...

Hi, @Branikolog!
I fixed the cut of single step move sounds and increased the sound fade time - it should reduce the clicks.
Sometimes at the end of movement there may be a case when the walk sound in the first channel is ending and then the sound in the second channel will be stopped (with fade). This is made not to interrupt the previously started walk sound (which is big enough as it has not still ended). If you think that we always have to play fully the last step sound I can make a change in the logic.

Also I forgot to mention that in this PR only walk sounds were changed (as they have step sound and sometimes a sequence of steps). Now the fly sounds use the same logic.

Could you please check how now the move sounds are played when you have time.

[Branikolog]

Hi, @Districh-ru !
For the flying creatures the first sound is played entirely, and all others are cut. I suppose it should work the opposite, right?

2023-06-21.13-48-00.mp4

Increased fade has fixed clicks for Air elementals. Except of the last sound: I can still hear a click when Elemental stops.

2023-06-21.13-48-20.mp4

[Districh-ru]

Hi, @Districh-ru ! For the flying creatures the first sound is played entirely, and all others are cut. I suppose it should work the opposite, right?

It works this way:

• The first sound always plays entirely in the 1st channel.
• If sound in the 1st channel is plying the next sound starts in the 2nd channel halting (with fade) the sound in the 2nd channel (if there is one already playing).
• If sound in the 1st channel is finished then the already playing sound in the 2nd channel is moved to the 1st and is played entirely.

While walking steps have their maximum volume at the beginning of sound (the sound of step), the flying sound has its maximum around the middle of the sound (before flapping the wing must be raised), so for some big birds on very high battle speeds the sound might be cut just before the sound of wing flap.
We can try to adjust it and set the minimum play time, or (I think it'll be better) just revert the changes for the flying creatures as flying sounds do not produce such mess as the walk sound: the fly sound contains only 1 wing flap, while the walk sounds mostly contains two or more steps.

Increased fade has fixed clicks for Air elementals. Except of the last sound: I can still hear a click when Elemental stops.

And again we are faced with the imperfection of SDL_mixer. :(
I thought that the sound fade-out will be a linear function of time like:

But in SDL_mixer it has the form of a step function of time:

And at every step, when the amplitude is significantly changed, there might be clicks in sound.
We need to find a way to make SDL_mixer to use less steps in fade effect or do a fade effect by our engine "on the fly" before performing the creature move, but it will increase the CPU load.

This issue needs some time to find a proper solution, so I'm marking this PR as draft.

[oleg-derevenetz]

But in SDL_mixer it has the form of a step function of time:

I believe, that's because SDL_mixer works this way: when SDL audio driver needs more data, the mixing function is called to produce another audio chunk, and the whole chunk has the same volume (I mean, the specific fading channel has the same volume during the whole chunk). We can try to reduce the chunkSize in the engine/audio.cpp from 2048 (current value) to 1024 or even 512, and this should help to make the curve "smoother". But this change should be properly tested on all supported platforms (e.g. on Android there were audio distortions when the chunk size was "too big").

[Districh-ru]

I believe, that's because SDL_mixer works this way: when SDL audio driver needs more data, the mixing function is called to produce another audio chunk, and the whole chunk has the same volume (I mean, the specific fading channel has the same volume during the whole chunk). We can try to reduce the chunkSize in the engine/audio.cpp from 2048 (current value) to 1024 or even 512, and this should help to make the curve "smoother". But this change should be properly tested on all supported platforms (e.g. on Android there were audio distortions when the chunk size was "too big").

@oleg-derevenetz, thanks for the suggestion. Yes, SDL_mixer works this way: for each chunk it adjusts the volume: Mix_Volume(i, (mix_channel[i].fade_volume * (mix_channel[i].fade_length-ticks)) / mix_channel[i].fade_length);.

I reduced the chunkSize to 1024 and increased the fade time a little and now the clicks are almost not noticeable (The spectrum analyzer shows them, but their level is low to hear). @Branikolog could you please check it when you have time.

Also I reverted the simultaneous sound limit for flying creatures.

I'll also check the changed chunkSize on Vita3K emulator.

[Districh-ru]

On Vita3K it seems to be OK.

Vita3K.2023-06-21.21-51-22-185.mp4

[Branikolog]

Hi, @Districh-ru !
I'm sorry for such a late response.

I've tested the build and found a few issues. Could you check, please, if you can fix it?

I've found that sometimes the last sound is faded too obviously. I've set battle speed as 5 and I can clearly hear fade in some cases.

2023-07-13.15-28-45.mp4

There's definitely a fade after the first crusader moves.
As the second crusader moves I can't hear fade clearly.
I've also found, that sometimes the last sound plays entirely, but sometimes it's cut (or at least it looks like):

2023-07-13.15-27-16.mp4

Here the first crusader walking sound looks like been interrupted. The second crusader has walking sound played completely after the troop has stopped.

When I set 10 speed some sounds are actually not really pleasant to ear. Like champion or crusader sounds.

2023-07-13.15-29-39.mp4

It sounds predictable according to the proposed logic in this PR, but it simply sounds not pleasant. I'm not sure, if it's possible to do something here... Maybe skip several simultaneous sounds or simply play only a single sound, since creature can cover the whole battlefield almost instantly...

[Districh-ru]

There's definitely a fade after the first crusader moves. As the second crusader moves I can't hear fade clearly.

Hi @Branikolog!
On the 1st video for the 1st crusader we hear the first sound fully and the second is faded because the first sound has not ended (the sound file has non ended. it is also possible that there was playing very silent sound from the end of the walk sound at that moment) when the crusader came to his destination. When the 2nd crusader walk for 3 steps we hear the first sound fully played, the second step sound is faded when the third step sound is started and this sound is not faded as the first sound has ended, so we can hear it fully.

Here the first crusader walking sound looks like been interrupted. The second crusader has walking sound played completely after the troop has stopped.

On the 2nd video we hear the fully played first sound. the walk sound for all other steps is interrupted. And the 1st crusader ends his movement when the first sound is near its and and we can clearly hear the fade of the second channel sound. The 2nd crusader has one step less movement and the first sound has more volume at this time than the faded sound in second channel.

Should we fade all sounds when the unit ends his movement (e.g. cavalry moves for 1 hexagon, but move sound is made for many steps, should we stop the sound when the unit stops)?

When I set 10 speed some sounds are actually not really pleasant to ear. Like champion or crusader sounds.
It sounds predictable according to the proposed logic in this PR, but it simply sounds not pleasant. I'm not sure, if it's possible to do something here... Maybe skip several simultaneous sounds or simply play only a single sound, since creature can cover the whole battlefield almost instantly...

Yes, in this case we have many-many sounds, but a little less than in master build as all second channel sounds are faded.
We cannot predict the movement time as it depends not only on the set battle speed, but on setting "VSYNC" (if it is off the engine can skip rendering frames if monitor frame rate is less the engine render rate), it also depends on CPU (slow devices compute moves slower and they take more time while sound is played on its default sample rate).
We can try to make playing only one sound for 10th speed, but some units move sounds consists of multiple steps, some of two and some of one. How it will look if creature moves for 6 hexagons and sounds like one step ...
I can not clearly decide how to behave here. And if we make this setting for 10th speed, should we do it fo 9th or 8th? :)

[oleg-derevenetz]

@Districh-ru FYI: libsdl-org/SDL_mixer#539

[drevoborod]

Issues I noticed:

• For creatures with 4 legs (Cavarly, Wolves, Unicorns) on both low and high speeds the last steps sound plays after a creature has been already stopped.
• On very fast flying creature (Phoenix under Haste spell) on maximum speed sound of the flight is being played after the flight had been finished.
• On high speeds (9, 10) Minotaurs make two step sound, and second one ends long after the creature finished moving.

[ihhub]

Hi @Districh-ru , could you please explain the status of this pull request and what should be done? Can we implement a part of logic / code which partially fixes the problem but doesn't introduce any issues?

[Districh-ru]

Hi, @ihhub,
a part of the logic was merged (#7471). On the weekend I'll check if we can quickly apply some logic from here.
But probably we should update each sound manually to allow their speed-up - some sounds have 1 step, others two or more; one steps are at the beginning of the sound, others - at the middle.