Android nnapi and IOS coreml providers support

[sitatec]

Hi, I would like to run kokoro/piper and nemo-fast-conformer-transducer-en-de-es-fr-14288 on GPU. I have built the Android libraries with API level 27, and initialized the models with provider: Platform.isAndroid ? "nnapi" : "coreml".
The ASR model seems to work, but not the TTS (currently piper). I'm getting this error:

W/sherpa-onnx(28096): Raw text: Hello User,
E/CpuExecutor(28096): Invalid dimensions for model operand: packages/modules/NeuralNetworks/common/types/src/TypeUtils.cpp:249: Incompatible ranks: [1] and [1, 1]
E/CpuExecutor(28096): GATHER failed.
E/CpuExecutor(28096): Invalid dimensions for model operand: packages/modules/NeuralNetworks/common/types/src/TypeUtils.cpp:249: Incompatible ranks: [1] and [1, 1]
E/CpuExecutor(28096): GATHER failed.
E/onnxruntime(28096):  [E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running Nnapi_17332806132420531993_0 node. Name:'NnapiExecutionProvider_Nnapi_17332806132420531993_0_0' Status Message: model.cc:195 Predict ResultCode: ANEURALNETWORKS_OP_FAILED
E/libc++abi(28096): terminating due to uncaught exception of type Ort::Exception: Non-zero status code returned while running Nnapi_17332806132420531993_0 node. Name:'NnapiExecutionProvider_Nnapi_17332806132420531993_0_0' Status Message: model.cc:195 Predict ResultCode: ANEURALNETWORKS_OP_FAILED
F/libc    (28096): Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 28495 (DartWorker), pid 28096 (initylabs.waico)
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'samsung/p3qsqw/p3q:13/TP1A.220624.014/G998USQS8EWF1:user/release-keys'
Revision: '20'
ABI: 'arm64'
Processor: '6'
Timestamp: 2025-08-02 17:51:50.516924083+0000
Process uptime: 142s
Cmdline: ai.buinitylabs.waico
pid: 28096, tid: 28495, name: DartWorker  >>> ai.buinitylabs.waico <<<
uid: 10542
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
Abort message: 'terminating due to uncaught exception of type Ort::Exception: Non-zero status code returned while running Nnapi_17332806132420531993_0 node. Name:'NnapiExecutionProvider_Nnapi_17332806132420531993_0_0' Status Message: model.cc:195 Predict ResultCode: ANEURALNETWORKS_OP_FAILED'
    x0  0000000000000000  x1  0000000000006f4f  x2  0000000000000006  x3  000000762cec45c0
    x4  646277641f736766  x5  646277641f736766  x6  646277641f736766  x7  7f7f7f7f7f7f7f7f
    x8  00000000000000f0  x9  00000079e1248bf8  x10 0000000000000001  x11 00000079e1289870
    x12 000000000e6d0be2  x13 0000126b7f45984e  x14 0032aa1544bc0591  x15 0000000034155555
    x16 00000079e12f1d70  x17 00000079e12cd5b0  x18 0000007416c26000  x19 0000000000006dc0
    x20 0000000000006f4f  x21 00000000ffffffff  x22 000000762cec46f0  x23 000000762cec4730
    x24 000000762cec47e0  x25 00000076dade5a30  x26 0000000000000000  x27 00000076fbca9f10
    x28 00000074b0b38000  x29 000000762cec4640
    lr  00000079e127a7a8  sp  000000762cec45a0  pc  00000079e127a7d4  pst 0000000000001000
backtrace:
      #00 pc 00000000000537d4  /apex/com.android.runtime/lib64/bionic/libc.so (abort+168) (BuildId: 870560a8376a70249f9e9a7b480cc02f)
      #01 pc 00000000001b1328  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #02 pc 00000000001b0c20  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #03 pc 00000000001b0ae0  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #04 pc 00000000001cb9c0  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #05 pc 00000000001cb934  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (__cxa_throw+128) (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #06 pc 00000000001f2694  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #07 pc 00000000002f60c0  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #08 pc 00000000002f5430  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #09 pc 00000000002f0888  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #10 pc 00000000002ef324  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #11 pc 00000000001d6d1c  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #12 pc 00000000001d6b70  /data/app/~~A01C3y992I314D-JBBWPmQ==/ai.buinitylabs.waico-ZXvc_CqkRU08ly1XaTel6w==/base.apk!libsherpa-onnx-c-api.so (SherpaOnnxOfflineTtsGenerate+40) (BuildId: affd2aef8899f3160865dc3150389e59ef3ce5e4)
      #13 pc 0000000000008004  [anon:dart-code]

It happened on a Samsung Galaxy S21 Ultra using the Dart API

[sitatec]

Kokoro is working, the issue happens with piper. I know piper is fast even on CPU, but I have other models running on CPU, so running piper also there is making everything slow.

[csukuangfj]

gpu in sherpa-onnx means nvidia gpu, not the gpu from apple.

---

Can you tell us which piper tts model you are using? Different piper models run at different speeds. In general, smaller models run faster.

[sitatec]

I'm using medium models, piper-en-hfc-female, piper-fr-mls. The problem is not the speed but the fact that it is not running with Android's NNAPI.