DeepSpeech2 使用导出模型pdmodel和pdiparams测试时预测时间比使用cfg和ckpt预测的还要慢很多

[lichuanqi]

Hello，我挑了5条测试音频使用deepspeech2测试时发现导出模型的预测时间在30s左右，比使用 cfg 和 ckpt 预测的11s还要慢很多。终端输出也确实显示了使用 gpu:0，不知道为啥很慢。而且导出的比不导出的还慢。

环境：

• Ubuntu 20.04
• python 3.7
• paddlepaddle 2.3.2
• CUDA 10.2
• cuDNN 7.6

模型：deepspeech2_online_wenetspeech_1.0.4
导出模型文件：from paddlespeech.s2t.exps.deepspeech2.model import DeepSpeech2ExportTester as ExportTester
导出模型结果：

Setup model!
2022-09-27 10:16:00.647 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:352 - Test Total Examples: 5
2022-09-27 10:16:00.649 | INFO     | paddlespeech.s2t.modules.ctc:_init_ext_scorer:187 - begin to initialize the external scorer for decoding
2022-09-27 10:16:00.958 | INFO     | paddlespeech.s2t.modules.ctc:_init_ext_scorer:197 - language model: is_character_based = 1, max_order = 5, dict_size = 0
2022-09-27 10:16:00.961 | INFO     | paddlespeech.s2t.modules.ctc:_init_ext_scorer:198 - end initializing scorer
W0927 10:16:02.442078 137610 rnn_kernel.cu.cc:243] If the memory space of the Input WeightList is not continuous, less efficient calculation will be called. Please call flatten_parameters() to make the input memory continuous.
W0927 10:16:02.443998 137610 rnn_kernel.cu.cc:243] If the memory space of the Input WeightList is not continuous, less efficient calculation will be called. Please call flatten_parameters() to make the input memory continuous.
2022-09-27 10:16:08.502 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_1
2022-09-27 10:16:08.504 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 他仅凭腰部的力量在泳道上下翻腾蛹动蛇行状如海豚一直以一头的优势领先
2022-09-27 10:16:08.504 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 仅凭腰部的力量在甬道上下翻腾涌动蛇形状如海豚一直以一头的优势领先
2022-09-27 10:16:08.518 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.121212
2022-09-27 10:16:08.518 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:388 - Error rate [cer] (1/?) = 0.121212
2022-09-27 10:16:14.018 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_0
2022-09-27 10:16:14.020 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 绿是阳春烟景大块文章的底色四月的林峦更是绿得鲜活秀媚诗意盎然
2022-09-27 10:16:14.020 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 律是杨春燕京大块文章的底色四月的更是绿的鲜活秀美诗意盎然
2022-09-27 10:16:14.030 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.266667
2022-09-27 10:16:14.031 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:388 - Error rate [cer] (2/?) = 0.190476
2022-09-27 10:16:19.489 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_4
2022-09-27 10:16:19.490 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 她看看夜己很深白天的炎热已给夜凉吹散吩咐大家各自安息明天继续玩乐
2022-09-27 10:16:19.490 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 他看看夜已很深白天的炎热夜凉吹散吩咐大家各自安息明天继续玩乐
2022-09-27 10:16:19.503 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.125000
2022-09-27 10:16:19.503 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:388 - Error rate [cer] (3/?) = 0.168421
2022-09-27 10:16:24.497 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_3
2022-09-27 10:16:24.498 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 菜做好了一碗清蒸武昌鱼一碗蕃茄炒鸡蛋一碗榨菜干子炒肉丝
2022-09-27 10:16:24.499 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 菜做好了一碗清蒸武昌鱼一碗番茄炒鸡蛋一碗榨菜干煸炒肉丝
2022-09-27 10:16:24.508 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.074074
2022-09-27 10:16:24.508 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:388 - Error rate [cer] (4/?) = 0.147541
2022-09-27 10:16:29.299 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_2
2022-09-27 10:16:29.300 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 企业依靠技术挖潜增效他负责全厂产品质量与技术培训成了厂里的大忙人
2022-09-27 10:16:29.301 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 企业依靠技术挖潜增效他负责全厂产品质量与技术培训成了厂里的大忙人
2022-09-27 10:16:29.301 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.000000
2022-09-27 10:16:29.302 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:388 - Error rate [cer] (5/?) = 0.116883
2022-09-27 10:16:29.357 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:395 - Test: epoch: 0, step: 0, Final error rate [cer] (5/5) = 0.116883
2022-09-27 10:16:29.361 | INFO     | paddlespeech.s2t.training.timer:__exit__:44 - Test/Decode Done: 0:00:28.713772

cfg 和 ckpt 预测文件：from paddlespeech.s2t.exps.deepspeech2.model import DeepSpeech2Tester as Tester
cfg 和 ckpt 预测输出：

2022-09-27 10:07:51.278 | INFO     | paddlespeech.s2t.utils.layer_tools:print_params:60 - Total parameters: 38.0, 164.71M elements.
2022-09-27 10:07:51.278 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:setup_model:146 - Setup model!
2022-09-27 10:07:56.522 | INFO     | paddlespeech.s2t.utils.checkpoint:load_parameters:117 - Rank 0: Restore model from configs/deepspeech2_online_wenetspeech_1.0.4/avg_10.pdparams
2022-09-27 10:07:56.526 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:297 - Test Total Examples: 5
2022-09-27 10:07:56.527 | INFO     | paddlespeech.s2t.modules.ctc:_init_ext_scorer:187 - begin to initialize the external scorer for decoding
2022-09-27 10:07:56.831 | INFO     | paddlespeech.s2t.modules.ctc:_init_ext_scorer:197 - language model: is_character_based = 1, max_order = 5, dict_size = 0
2022-09-27 10:07:56.832 | INFO     | paddlespeech.s2t.modules.ctc:_init_ext_scorer:198 - end initializing scorer
2022-09-27 10:07:59.325 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_1
2022-09-27 10:07:59.326 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 他仅凭腰部的力量在泳道上下翻腾蛹动蛇行状如海豚一直以一头的优势领先
2022-09-27 10:07:59.326 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 仅凭腰部的力量在甬道上下翻腾涌动蛇形状如海豚一直以一头的优势领先
2022-09-27 10:07:59.339 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.121212
2022-09-27 10:07:59.340 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:322 - Error rate [cer] (1/?) = 0.121212
2022-09-27 10:08:00.198 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_0
2022-09-27 10:08:00.198 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 绿是阳春烟景大块文章的底色四月的林峦更是绿得鲜活秀媚诗意盎然
2022-09-27 10:08:00.199 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 律是杨春燕京大块文章的底色四月的更是绿的鲜活秀美诗意盎然
2022-09-27 10:08:00.209 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.266667
2022-09-27 10:08:00.210 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:322 - Error rate [cer] (2/?) = 0.190476
2022-09-27 10:08:00.981 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_4
2022-09-27 10:08:00.982 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 她看看夜己很深白天的炎热已给夜凉吹散吩咐大家各自安息明天继续玩乐
2022-09-27 10:08:00.982 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 他看看夜已很深白天的炎热夜凉吹散吩咐大家各自安息明天继续玩乐
2022-09-27 10:08:00.994 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.125000
2022-09-27 10:08:00.995 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:322 - Error rate [cer] (3/?) = 0.168421
2022-09-27 10:08:01.753 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_3
2022-09-27 10:08:01.754 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 菜做好了一碗清蒸武昌鱼一碗蕃茄炒鸡蛋一碗榨菜干子炒肉丝
2022-09-27 10:08:01.755 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 菜做好了一碗清蒸武昌鱼一碗番茄炒鸡蛋一碗榨菜干煸炒肉丝
2022-09-27 10:08:01.770 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.074074
2022-09-27 10:08:01.770 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:322 - Error rate [cer] (4/?) = 0.147541
2022-09-27 10:08:02.450 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:276 - Utt: A2_2
2022-09-27 10:08:02.451 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:277 - Ref: 企业依靠技术挖潜增效他负责全厂产品质量与技术培训成了厂里的大忙人
2022-09-27 10:08:02.451 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:278 - Hyp: 企业依靠技术挖潜增效他负责全厂产品质量与技术培训成了厂里的大忙人
2022-09-27 10:08:02.452 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:compute_metrics:281 - Current error rate [cer] = 0.000000
2022-09-27 10:08:02.453 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:322 - Error rate [cer] (5/?) = 0.116883
2022-09-27 10:08:02.494 | INFO     | paddlespeech.s2t.exps.deepspeech2.model:test:330 - Test: epoch: 0, step: 0, Final error rate [cer] (5/5) = 0.116883
2022-09-27 10:08:02.497 | INFO     | paddlespeech.s2t.training.timer:__exit__:44 - Test/Decode Done: 0:00:11.218123

[zh794390558]

不明太你说的导出和不导出是指什么？是说模型转成静态图和动态图的做对比吗？
有一个提示，可以在paddle的issue里问下怎么处理。

W0927 10:16:02.442078 137610 rnn_kernel.cu.cc:243] If the memory space of the Input WeightList is not continuous, less efficient calculation will be called. Please call flatten_parameters() to make the input memory continuous.

[lichuanqi]

是静态图和动态图的对比。
好的，感谢回复。我再根据这个错误查查资料，测试的时候GPU占用的显存很少，估计有点问题。

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue is closed. Please re-open if needed.