[Optim] Optimize some long-tail cases. #861

Job	Run time
Pre-check on Ubuntu-latest	25s
Run unit tests on ubuntu-latest	3m 25s
Run unit tests on ubuntu-latest	3m 11s
Run unit tests on ubuntu-24.04-arm	1m 56s
Run unit tests on ubuntu-latest	3m 30s
Run unit tests on ubuntu-24.04-arm	1m 59s
Run unit tests on ubuntu-24.04-arm	1m 52s
Run unit tests on ubuntu-24.04-arm	1m 55s
Run unit tests on ubuntu-24.04-arm	2m 2s
Run unit tests on ubuntu-latest	3m 13s
Run unit tests on ubuntu-latest	3m 15s
Run unit tests on ubuntu-24.04-arm	2m 2s
Run unit tests on ubuntu-latest	3m 46s
Run unit tests on windows-latest	6m 2s
Run unit tests on windows-latest	6m 30s
Run unit tests on windows-latest	5m 8s
Run unit tests on windows-latest	5m 51s
Run unit tests on windows-latest	6m 15s
Run unit tests on windows-latest	7m 10s
Run unit tests on macos-13	3m 27s
Run unit tests on macos-14	2m 21s
Run unit tests on macos-13	3m 38s
Run unit tests on macos-13	3m 16s
Run unit tests on macos-14	2m 13s
Run unit tests on macos-14	2m 3s
Run unit tests on macos-13	3m 46s
Run unit tests on macos-14	2m 0s
Run unit tests on macos-14	2m 21s
Run unit tests on macos-13	3m 9s
Run unit tests on macos-14	2m 9s
	1h 39m 50s

Provide feedback