Skip to content

编译runtime/webscoket,使用GPU并且禁止torch_blade,编译通过会运行报错异常 #2767

@lukeewin

Description

@lukeewin

1. 环境

OS: ubuntu 24.04
Python: 3.8
CUDA: 12.8
PyTorch: 1.12.1+cu113

2. 复现步骤

vim FunASR/runtime/websocket/CMakeLists.txt

修改下面内容

option(GPU "Whether to build with GPU" OFF) --> option(GPU "Whether to build with GPU" ON)
set(TORCH_DIR "/home/lukeewin/miniconda3/envs/funasr_cpp/lib/python3.8/site-packages/torch") --> 修改为自己服务器中的torch路径
set(TORCH_BLADE_DIR "/home/lukeewin/miniconda3/envs/funasr_cpp/lib/python3.8/site-packages/torch_blade") --> 注释掉
link_directories(${TORCH_DIR}/lib) --> 注释掉
vim FunASR/runtime/onnxruntime/CMakeLists.txt
set(TORCH_DIR "/home/lukeewin/miniconda3/envs/funasr_cpp/lib/python3.8/site-packages/torch") --> 修改为自己服务器中的torch的路径
set(TORCH_BLADE_DIR "/home/lukeewin/miniconda3/envs/funasr_cpp/lib/python3.8/site-packages/torch_blade") --> 注释掉
link_directories(${TORCH_BLADE_DIR}) --> 注释掉
vim FunASR/runtime/onnxruntime/src/CMakeLists.txt
set(TORCH_DEPS torch torch_cuda torch_cpu c10 c10_cuda torch_blade ral_base_context) --> set(TORCH_DEPS torch torch_cuda torch_cpu c10 c10_cuda)

然后执行下面命令编译,可以正常通过编译

cd FunASR/runtime/websocket
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=release .. -DONNXRUNTIME_DIR=/home/lukeewin/Projects/FunASR_New/FunASR/runtime/websocket/onnxruntime-linux-x64-1.14.0 -DFFMPEG_DIR=/home/lukeewin/Projects/FunASR_New/FunASR/runtime/websocket/ffmpeg-master-latest-linux64-gpl-shared
make -j28

但是在执行这个funasr-wss-server可执行程序的时候会报错下面的错误,下面是我的执行命令参数

./funasr-wss-server --batch-size 32 --gpu --model-thread-num 16 --decoder-thread-num 16 --io-thread-num 8 --port 8000 --listen-ip '0.0.0.0' --lm-revision 'master' --lm-dir '/home/lukeewin/Projects/FunASR_New/models/iic/speech_ngram_lm_zh-cn-ai-wesp-fst' --itn-revision 'master' --itn-dir '/home/lukeewin/Projects/FunASR_New/models/thuduj12/fst_itn_zh' --punc-quant true --punc-revision 'master' --punc-dir '/home/lukeewin/Projects/FunASR_New/models/iic/punc_ct-transformer_cn-en-common-vocab471067-large-onnx' --vad-quant true  --vad-revision 'master' --vad-dir '/home/lukeewin/Projects/FunASR_New/models/iic/speech_fsmn_vad_zh-cn-16k-common-onnx' --bladedisc false --quantize true --model-revision 'master'  --model-dir '/home/lukeewin/Projects/FunASR_New/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-torchscript'  --certfile ''

执行中途会报错下面的错误

I20260101 13:13:09.046938  6587 funasr-wss-server.cpp:25] model-dir : /home/lukeewin/Projects/FunASR_New/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-torchscript
I20260101 13:13:09.046972  6587 funasr-wss-server.cpp:25] quantize : true
I20260101 13:13:09.046974  6587 funasr-wss-server.cpp:25] bladedisc : false
I20260101 13:13:09.046976  6587 funasr-wss-server.cpp:25] vad-dir : /home/lukeewin/Projects/FunASR_New/models/iic/speech_fsmn_vad_zh-cn-16k-common-onnx
I20260101 13:13:09.046978  6587 funasr-wss-server.cpp:25] vad-quant : true
I20260101 13:13:09.046980  6587 funasr-wss-server.cpp:25] punc-dir : /home/lukeewin/Projects/FunASR_New/models/iic/punc_ct-transformer_cn-en-common-vocab471067-large-onnx
I20260101 13:13:09.046983  6587 funasr-wss-server.cpp:25] punc-quant : true
I20260101 13:13:09.046985  6587 funasr-wss-server.cpp:25] itn-dir : /home/lukeewin/Projects/FunASR_New/models/thuduj12/fst_itn_zh
I20260101 13:13:09.046988  6587 funasr-wss-server.cpp:25] lm-dir : /home/lukeewin/Projects/FunASR_New/models/iic/speech_ngram_lm_zh-cn-ai-wesp-fst
I20260101 13:13:09.046989  6587 funasr-wss-server.cpp:25] hotword : /workspace/resources/hotwords.txt
I20260101 13:13:09.046991  6587 funasr-wss-server.cpp:25] model-revision : master
I20260101 13:13:09.046993  6587 funasr-wss-server.cpp:25] vad-revision : master
I20260101 13:13:09.046995  6587 funasr-wss-server.cpp:25] punc-revision : master
I20260101 13:13:09.046998  6587 funasr-wss-server.cpp:25] itn-revision : master
I20260101 13:13:09.046999  6587 funasr-wss-server.cpp:25] lm-revision : master
I20260101 13:13:10.303433  6587 funasr-wss-server.cpp:235] Set vad-dir : /home/lukeewin/Projects/FunASR_New/models/iic/speech_fsmn_vad_zh-cn-16k-common-onnx
I20260101 13:13:11.547780  6587 funasr-wss-server.cpp:316] Set model-dir : /home/lukeewin/Projects/FunASR_New/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-torchscript
I20260101 13:13:12.684445  6587 funasr-wss-server.cpp:358] Set itn-dir : /home/lukeewin/Projects/FunASR_New/models/thuduj12/fst_itn_zh
I20260101 13:13:13.910359  6587 funasr-wss-server.cpp:400] Set lm-dir : /home/lukeewin/Projects/FunASR_New/models/iic/speech_ngram_lm_zh-cn-ai-wesp-fst
I20260101 13:13:15.146636  6587 funasr-wss-server.cpp:437] Set punc-dir : /home/lukeewin/Projects/FunASR_New/models/iic/punc_ct-transformer_cn-en-common-vocab471067-large-onnx
I20260101 13:13:15.146659  6587 funasr-wss-server.cpp:466] hotword path: /workspace/resources/hotwords.txt
E20260101 13:13:15.146682  6587 util.cpp:1009] Unable to open hotwords file: /workspace/resources/hotwords.txt. If you have not set hotwords, please ignore this message.
I20260101 13:13:15.147305  6587 funasr-wss-server.cpp:497] SSL is closed!
I20260101 13:13:15.160056  6587 fsmn-vad.cpp:58] Successfully load model from /home/lukeewin/Projects/FunASR_New/models/iic/speech_fsmn_vad_zh-cn-16k-common-onnx/model_quant.onnx
I20260101 13:13:15.268329  6587 paraformer-torch.cpp:41] CUDA is available, running on GPU
E20260101 13:13:16.417017  6587 paraformer-torch.cpp:62] Error when load am model: /home/lukeewin/Projects/FunASR_New/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-torchscript/model.torchscript
Unknown type name '__torch__.torch.classes.torch_blade.Engine':
  File "code/__torch__/funasr/models/sanm/encoder/___torch_mangle_645.py", line 4
  __parameters__ = []
  __buffers__ = []
  disc_grp0_len5729_0 : __torch__.torch.classes.torch_blade.Engine
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
  def forward(self: __torch__.funasr.models.sanm.encoder.___torch_mangle_645.SANMEncoderExport,
    speech: Tensor,

请问有人遇到过这个问题吗?是如何解决的呢?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions