把Sherpa-onnx部署到安卓端的3568

1.下载Sherpa-onnx源码git clone https://github.com/k2-fsa/sherpa-onnx2.准备编译//先设置ndk地址,你们根据你们实际的ndl地址配置即可export ANDROID_NDK../../../android-ndk-r26d//设置编译rknnexport SHERPA_ONNX_ENABLE_RKNNON//执行sherpa-onnx自带的脚步./build-android-arm64-v8a.sh等待,脚本会自动安装所有的内容,然后编译出对应的执行库,这里rknn的库也会编译出来3.把代码导入android studio从第一步的源码找到sherpa-onnx\android\SherpaOnnx项目,直接导入对应的android studio,这里我是把第三步的库放进来了,这里我是arm64-v8a,我在assets里面放入了我的rknn模型4.下载双语模型,官网是已经转化好了的我这里是使用命令下载的,你们可以直接去官网找到你们要的内容//下载wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-rk3568-streaming-zipformer-bilingual-zh-en-2023-02-20.tar.bz2//解压tar -jxvf sherpa-onnx-rk3568-streaming-zipformer-bilingual-zh-en-2023-02-20.tar.bz2这是模型官网地址:发布ASR-models·K2-FSA/夏尔巴-ONNX ·GitHub5.修改代码,源码自带是没有rknn的,只有一个qnn我们在activity里面新增代码,我直接把这一段全贴出来var assetManager: AssetManager? application.assets if (config.modelConfig.provider qnn) { Log.i(TAG, nativelibdir: ${applicationInfo.nativeLibraryDir}) OnlineRecognizer.prependAdspLibraryPath(applicationInfo.nativeLibraryDir) val transducer config.modelConfig.transducer val qnnConfig transducer.qnnConfig if (qnnConfig.backendLib.isEmpty()) { throw IllegalArgumentException(You should provide libQnnHtp.so for qnn) } config.modelConfig.tokens copyAssetToInternalStorage(config.modelConfig.tokens, this) if (transducer.encoder.isNotEmpty()) { transducer.encoder copyAssetToInternalStorage(transducer.encoder, this) } if (transducer.decoder.isNotEmpty()) { transducer.decoder copyAssetToInternalStorage(transducer.decoder, this) } if (transducer.joiner.isNotEmpty()) { transducer.joiner copyAssetToInternalStorage(transducer.joiner, this) } if (qnnConfig.contextBinary.isNotEmpty()) { qnnConfig.contextBinary copyAssetListToInternalStorage(qnnConfig.contextBinary, this) } if (config.hr.lexicon.isNotEmpty()) { config.hr.lexicon copyAssetToInternalStorage(config.hr.lexicon, this) } if (config.hr.ruleFsts.isNotEmpty()) { config.hr.ruleFsts copyAssetToInternalStorage(config.hr.ruleFsts, this) } assetManager null }else if (config.modelConfig.provider rknn) { // RKNN 新增分支 Log.i(TAG, Enter RKNN provider file copy logic) val transducer config.modelConfig.transducer // 1. 拷贝tokens词表文件 config.modelConfig.tokens copyAssetToInternalStorage(config.modelConfig.tokens, this) // 2. 拷贝三个rknn模型文件 encoder/decoder/joiner if (transducer.encoder.isNotEmpty()) { transducer.encoder copyAssetToInternalStorage(transducer.encoder, this) Log.i(TAG, Copy encoder.rknn success: ${transducer.encoder}) } if (transducer.decoder.isNotEmpty()) { transducer.decoder copyAssetToInternalStorage(transducer.decoder, this) Log.i(TAG, Copy decoder.rknn success: ${transducer.decoder}) } if (transducer.joiner.isNotEmpty()) { transducer.joiner copyAssetToInternalStorage(transducer.joiner, this) Log.i(TAG, Copy joiner.rknn success: ${transducer.joiner}) } // 3. 拷贝同音替换词典、文法文件和QNN逻辑一致 if (config.hr.lexicon.isNotEmpty()) { config.hr.lexicon copyAssetToInternalStorage(config.hr.lexicon, this) } if (config.hr.ruleFsts.isNotEmpty()) { config.hr.ruleFsts copyAssetToInternalStorage(config.hr.ruleFsts, this) } // RKNN不需要assetManager置空和QNN保持统一 assetManager null }在OnlineRecognizer.kt里面也把我们下载的模型配置进去,上面有示例,基本只要改下文件夹名字就行这里我们新增了一个1002,我们再把mainactivity的type类型改一下,让代码执行1002的配置,源码默认是0我们运行一遍使用,基本都能识别出来,我手上的板子npu算力不够,跑一段时间就挂了,但是能用.基本上一运行,npu就拉满了,你们如果要用,最好是1T以上的算力,会好点.