[Bug] 在JetsonAGX Orin上 mmdet模型转换失败但是mmpose转换成功, mmdeploy/csrc/mmdeploy/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp报错

[liuky74]

Checklist

• I have searched related issues but cannot get the expected help.
• 2. I have read the FAQ documentation but cannot get the expected help.
• 3. The bug has not been fixed in the latest version.

Describe the bug

经过一番尝试, 我在Jetson Orin AGX 64G(cuda 12.2, TensorRT 8.6.2.3 Torch2.3, mmcv=2.1, onnx=1.16)版本上成功编译了mmdeploy并且转换成功了mmpose训练的模型并通过了验证, 我的模型训练环境为 cuda11.8, cudnn=8.9, tensorRT=8.6.1.6, torch=2.0.1

但是在尝试 转换mmdet的训练模型时报错, 根据报错信息模型已经转换成功, 但是在执行"visualize_model"可视化函数验证的时候出错, 根据报错信息如下:
#assertion/home/smartgis/workspace/project/mmdeploy/csrc/mmdeploy/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp,103
我查看了这个文件, 显示如下:

由于我的C++水平一般, 所以无法继续深入查找原因了, 还希望得到各位的帮助, 我的整个流程就差这最后一步了, 谢谢!

另外我还测试了转为cpu版本的onnx模型, 也能通过测试

---

After some attempts, I set up the Jetson Orin AGX 64G(cuda 12.2, TensorRT 8.6.2.3 Torch2.3, mmcv=2.1, On onnx=1.16), mmdeploy has been successfully compiled and the mmpose training model has been successfully converted and verified. My model training environment is cuda11.8, cudnn=8.9, tensorRT=8.6.1.6, torch=2.0.1

But in an attempt to convert mmdet training model times wrong, According to the error message, the model has been successfully converted, However, an error occurred while executing the "visualize_model" function, according to the error message as follows:
`#assertion/home/smartgis/workspace/project/mmdeploy/csrc/mmdeploy/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp, 103 '
I have viewed this file and it shows the following:

because of my c + + level is general, so unable to further find the reason, I also hope to get your help, my whole process is missing this last step, thank you!

Reproduction

python tools/deploy.py

Environment

11/22 23:30:26 - mmengine - INFO - TorchVision: 0.18.0a0+6043bc2
11/22 23:30:26 - mmengine - INFO - OpenCV: 4.10.0
11/22 23:30:26 - mmengine - INFO - MMEngine: 0.10.5
11/22 23:30:26 - mmengine - INFO - MMCV: 2.1.0
11/22 23:30:26 - mmengine - INFO - MMCV Compiler: GCC 10.5
11/22 23:30:26 - mmengine - INFO - MMCV CUDA Compiler: 12.2
11/22 23:30:26 - mmengine - INFO - MMDeploy: 1.3.1+3f8604b
11/22 23:30:26 - mmengine - INFO - 

11/22 23:30:26 - mmengine - INFO - **********Backend information**********
11/22 23:30:26 - mmengine - INFO - tensorrt:    8.6.2
11/22 23:30:26 - mmengine - INFO - tensorrt custom ops: Available
11/22 23:30:26 - mmengine - INFO - ONNXRuntime: None
11/22 23:30:26 - mmengine - INFO - ONNXRuntime-gpu:     1.16.0
11/22 23:30:26 - mmengine - INFO - ONNXRuntime custom ops:      NotAvailable
11/22 23:30:26 - mmengine - INFO - pplnn:       None
11/22 23:30:26 - mmengine - INFO - ncnn:        None
11/22 23:30:26 - mmengine - INFO - snpe:        None
11/22 23:30:26 - mmengine - INFO - openvino:    None
11/22 23:30:26 - mmengine - INFO - torchscript: 2.3.0
11/22 23:30:26 - mmengine - INFO - torchscript custom ops:      NotAvailable
11/22 23:30:26 - mmengine - INFO - rknn-toolkit:        None
11/22 23:30:26 - mmengine - INFO - rknn-toolkit2:       None
11/22 23:30:26 - mmengine - INFO - ascend:      None
11/22 23:30:26 - mmengine - INFO - coreml:      None
11/22 23:30:26 - mmengine - INFO - tvm: None
11/22 23:30:26 - mmengine - INFO - vacc:        None
11/22 23:30:26 - mmengine - INFO - 

11/22 23:30:26 - mmengine - INFO - **********Codebase information**********
11/22 23:30:27 - mmengine - INFO - mmdet:       3.3.0
11/22 23:30:27 - mmengine - INFO - mmseg:       None
11/22 23:30:27 - mmengine - INFO - mmpretrain:  None
11/22 23:30:27 - mmengine - INFO - mmocr:       None
11/22 23:30:27 - mmengine - INFO - mmagic:      None
11/22 23:30:27 - mmengine - INFO - mmdet3d:     None
11/22 23:30:27 - mmengine - INFO - mmpose:      1.3.2
11/22 23:30:27 - mmengine - INFO - mmrotate:    None
11/22 23:30:27 - mmengine - INFO - mmaction:    None
11/22 23:30:27 - mmengine - INFO - mmrazor:     None
11/22 23:30:27 - mmengine - INFO - mmyolo:      None

Error traceback

11/22 23:31:37 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/22 23:31:37 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/22 23:31:37 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/22 23:31:37 - mmengine - INFO - Successfully loaded tensorrt plugins from /home/smartgis/workspace/project/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
11/22 23:31:37 - mmengine - INFO - Successfully loaded tensorrt plugins from /home/smartgis/workspace/project/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
#assertion/home/smartgis/workspace/project/mmdeploy/csrc/mmdeploy/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp,103