IB-Robot BQ3588HM板端部署OpenHarmonyRKNN NPU推理全攻略【免费下载链接】IB_RobotSave the code of IB-Robot, an AI robot execution framework developed by openEuler Embedded for embodied intelligence scenarios. It includes references to the forked version of tensormsg, references to lerobot fork, code references to the lerobot_ros2:ros2_ws branch, as well as some code related to development usability.项目地址: https://gitcode.com/openeuler/IB_Robot前往项目官网免费下载https://ar.openeuler.org/ar/IB-Robot作为openEuler Embedded开发的具身智能机器人执行框架为AI机器人开发者提供了完整的BQ3588HM板端部署解决方案。本文将详细介绍如何在Rockchip RK3588芯片的BQ3588HM开发板上基于OpenHarmony系统实现端到端的RKNN NPU推理全链路部署让您的机器人拥有强大的边缘计算能力 为什么选择BQ3588HM进行机器人AI部署BQ3588HM开发板搭载的RK3588芯片拥有6 TOPS的NPU算力结合OpenHarmony系统的实时性优势为机器人AI应用提供了理想的硬件平台。通过IB-Robot框架您可以轻松实现边缘AI推理直接在板端运行ACT、Diffusion Policy等视觉-动作策略模型实时控制20Hz的高频控制循环确保机器人动作流畅自然多传感器融合同时处理多路摄像头、IMU等传感器数据自主运行脱离主机依赖实现真正的端侧智能 部署前准备清单在开始部署前请确保您已准备好以下环境组件要求说明主机环境Ubuntu 22.04, Python 3.10用于交叉编译和模型转换开发板Bearkey BQ3588HM已烧录OpenHarmony EmbodiedAI 1.0.1系统网络连接HDC over TCP开发板IP:192.168.136.111:8710模型文件训练好的ACT策略checkpoint如models/502000/目录工具链HDC调试工具路径/home/xqw/Research/oh_sdk/toolchains/hdc 第一步主机端模型转换ONNX → RKNN1.1 创建专用RKNN虚拟环境由于rknn-toolkit2与LeRobot的依赖冲突需要单独创建虚拟环境python3 -m venv .venv-rknn source .venv-rknn/bin/activate pip install rknn-toolkit22.3.21.2 一键转换模型使用项目自带的转换脚本轻松完成模型转换# 在项目根目录下 source .venv-rknn/bin/activate python src/model_utils/model_utils/export_onnx_rknn.py \ --policy_path models/502000 \ --output models/502000/act_ros2_rknn.rknn \ --dtype float16转换结果生成约114MB的act_ros2_rknn.rknn文件支持两个480x640相机输入和14维状态向量。⚙️ 第二步板端环境配置2.1 板端Python环境信息环境项配置值Python版本3.12musl libc扩展模块后缀.cpython-312-aarch64-linux-ohos.sosite-packages路径/sys_prod/robot/out/lib/python3.12/site-packages/RKNN运行时/vendor/lib64/librknnrt.sov2.4.1b02.2 关键配置修复修复rknnlite .so文件后缀# 重命名所有-gnu.so为-ohos.so $HDC_BIN -t $HDC_TARGET shell for f in /sys_prod/robot/out/lib/python3.12/site-packages/rknnlite/api/*.cpython-312-aarch64-linux-gnu.so; do new${f%-gnu.so}-ohos.so; cp $f $new; done配置librknnrt.so路径# 挂载根文件系统为可写 $HDC_BIN -t $HDC_TARGET shell mount -o rw,remount / $HDC_BIN -t $HDC_TARGET shell mkdir -p /usr/lib $HDC_BIN -t $HDC_TARGET shell cp /vendor/lib64/librknnrt.so /usr/lib/设置LD_PRELOAD解决符号问题export LD_PRELOAD/sys_prod/robot/out/lib/libpython3.12.so.1.0 第三步部署模型并运行推理3.1 推送模型到开发板$HDC_BIN -t $HDC_TARGET file send models/502000/act_ros2_rknn.rknn /data/local/tmp/act_ros2_rknn.rknn3.2 运行NPU推理测试import numpy as np import time from rknnlite.api import RKNNLite # 初始化RKNN rknn RKNNLite() rknn.load_rknn(/data/local/tmp/act_ros2_rknn.rknn) rknn.init_runtime(targetNone) # 使用本机NPU # 准备测试数据 state np.random.randn(1, 14).astype(np.float32) cam_high np.random.randn(1, 3, 480, 640).astype(np.float32) cam_left np.random.randn(1, 3, 480, 640).astype(np.float32) # 执行推理 t0 time.time() outputs rknn.inference(inputs[state, cam_high, cam_left]) t1 time.time() print(f输出shape: {outputs[0].shape}) # (1, 100, 6) print(f推理耗时: {t1 - t0:.3f}s) # RK3588 NPU约121ms rknn.release()一键验证命令$HDC_BIN -t $HDC_TARGET shell LD_PRELOAD/sys_prod/robot/out/lib/libpython3.12.so.1.0 python3 -c import numpy as np, time from rknnlite.api import RKNNLite rknn RKNNLite() rknn.load_rknn(\/data/local/tmp/act_ros2_rknn.rknn\) rknn.init_runtime(targetNone) state np.random.randn(1, 14).astype(np.float32) cam_high np.random.randn(1, 3, 480, 640).astype(np.float32) cam_left np.random.randn(1, 3, 480, 640).astype(np.float32) t0 time.time() outputs rknn.inference(inputs[state, cam_high, cam_left]) print(f\output: shape{outputs[0].shape}, time{time.time()-t0:.3f}s\) rknn.release() 第四步单板全链路闭环部署4.1 整体架构设计IB-Robot在BQ3588HM上实现完整的摄像头采集 → RKNN NPU推理 → 机械臂控制闭环BQ3588HM 开发板 (RK3588, OpenHarmony) ├── usb_cam_node_exe × 2 (top wrist 相机) ├── static_transform_publisher × 4 (TF坐标变换) ├── lerobot_policy_node × 1 (RKNN NPU推理, ACT策略) ├── action_dispatcher_node × 1 (动作分发, 20Hz) └── so101_hardware (ros2_control, /dev/ttyACM0) ↳ arm_position_controller / gripper_position_controller4.2 内核驱动编译关键步骤开发板默认内核缺少SO-101机械臂和游戏手柄所需的USB驱动需要重新编译内核获取OpenHarmony源码下载EmbodiedAI 1.0.1源码到/data/oh_build修改内核配置编辑kernel/linux/config/linux-6.6/rk3588/arch/arm64_defconfig确保以下选项启用CONFIG_USB_ACMy # SO-101机械臂必需 CONFIG_USB_SERIAL_CH341y CONFIG_INPUT_JOYDEVy # 游戏手柄支持 CONFIG_JOYSTICK_XPADy编译并刷入内核cd /data/oh_build ./build.sh -p bq3588 --ccache dd ifout/bq3588/packages/phone/images/boot_linux.img of/dev/block/by-name/boot_linux reboot4.3 机械臂校准首次使用前必须执行校准# SSH登录板端 ssh root192.168.136.111 # 加载环境 cd /data . ./ros2ohos.env . /data/ibrobot/install/setup.sh export RMW_IMPLEMENTATIONrmw_cyclonedds_cpp # 执行校准 ros2 run so101_hardware calibrate_arm --arm follower --port /dev/ttyACM0校准文件保存在~/.calibrate/so101_follower_calibrate.json。️ 第五步交叉编译第三方ROS包5.1 编译usb_cam包板端预装的usb_cam缺少必要依赖需要使用OH交叉编译工具链重新编译# Docker交叉编译环境 docker run --rm -i \ -e WS_ROOT/mnt/ohos/tmp \ -e OHOS_SDK/mnt/ohos/tmp/ohos-robot-toolchain/18 \ --name ibrobot-oh-build \ -v ${OH_CUSTOM_ROOT}:/mnt/ohos \ -v ${OH_CUSTOM_ROOT}:/mnt/ohos/tmp \ voxelsky/ohos-ros-humble-builder:v0.1.5 \ bash -lc set -euo pipefail export OHOS_CPUaarch64 export OHOS_SDK/mnt/ohos/tmp/ohos-robot-toolchain/18 build-ros-humble --custom \ --wd /mnt/ohos/tmp/ibrobot_oh_ws \ --custom-prefix /data/ibrobot/install \ --colcon-args --packages-select usb_cam 5.2 部署Python依赖包对于纯Python包可以直接复制到板端site-packages包名版本用途deepdiff7.0.1配置对比draccus0.10.0LeRobot配置框架scservo_sdk-飞特舵机SDKordered-set4.1.0deepdiff依赖⚡ 第六步板端多运行时环境配置6.1 完整环境变量设置板端需要同时运行ROS 2节点和PyTorch推理必须正确设置环境变量# 第一步设置Python运行时 export PYTHONHOME/data/local/skh-run/usr export LD_PRELOAD${PYTHONHOME}/lib/libpython3.12.so.1.0:${PYTHONHOME}/lib/libomp.so export LD_LIBRARY_PATH${PYTHONHOME}/lib:/sys_prod/robot/out/lib:/data/install/lib:/vendor/lib64 export PYTHONPATH${PYTHONHOME}/lib/python3.12/site-packages:/sys_prod/robot/out/lib/python3.12/site-packages:/data/install/lib/python3.12/site-packages # 第二步加载ROS IB Robot环境 cd /data . ./ros2ohos.env . /data/ibrobot/install/setup.sh export RMW_IMPLEMENTATIONrmw_cyclonedds_cpp # 必须使用CycloneDDS # 第三步补充IB Robot路径 export PATH${PYTHONHOME}/bin:$PATH export PYTHONPATH/data/ibrobot/install/lerobot/src:/data/ibrobot/install/inference_service/lib/python3.12/site-packages:${PYTHONPATH} export LD_LIBRARY_PATH/data/ibrobot/install/ibrobot_msgs/lib:/data/ibrobot/install/inference_service/lib:${PYTHONHOME}/lib/python3.12/site-packages/torch/lib:${LD_LIBRARY_PATH}关键提示PYTHONHOMELD_PRELOAD必须在source setup.sh之前设置6.2 RKNN模式跳过LeRobot PreprocessorIB-Robot已针对RKNN模式优化跳过不必要的预处理# src/inference_service/inference_service/core/coordinator.py is_rknn self._device_name rknn if is_rknn: # RKNN: 使用基础TensorPreprocessor self._preprocessor TensorPreprocessor(deviceself._device) self._postprocessor TensorPostprocessor(deviceself._device) 第七步完整启动与验证7.1 清理残留进程# 每次重启前清理 pkill -9 -f ros2 launch\|lerobot_policy_node\|action_dispatcher_node\|usb_cam_node_exe\|static_transform_publisher7.2 启动全链路系统# SSH登录板端并设置环境 ssh root192.168.136.111 # 设置完整环境变量如6.1节所示 # ... # 启动机器人系统 ros2 launch robot_config robot.launch.py \ robot_config:so101_single_arm \ use_sim:false \ control_mode:model_inference \ 21 | tee /data/launch.log7.3 预期输出[usb_cam_node_exe-1] [INFO] [top_camera]: Starting top (/dev/video20) at 640x480 via mmap (mjpeg2rgb) at 30 FPS [lerobot_policy_node-7] [INFO] [act_inference_node]: Using inference_backendrknn, tensor_devicecpu [lerobot_policy_node-7] [INFO] [act_inference_node]: ✓ First inference complete (monolithic): total~500ms [action_dispatcher_node-8] [INFO] [action_dispatcher]: ✓ First inference received: chunk100 性能数据与优化建议性能基准指标数值说明NPU推理延迟首次~900ms包含模型加载时间NPU推理延迟稳定~500ms持续推理延迟总端到端延迟~520ms含预处理时间Action chunk size100每次推理生成的动作块数控制频率20 Hz机器人控制频率相机帧率top: 30 FPS, wrist: 60 FPS双摄像头配置模型大小114 MBfloat16格式优化建议模型优化使用--dtype float16减少模型大小输入优化确保输入数据格式正确NCHW格式环境隔离使用专用虚拟环境避免依赖冲突内存管理定期清理RKNN实例释放NPU内存 常见问题排查指南问题现象可能原因解决方案/dev/ttyACM0不存在内核缺少CONFIG_USB_ACMy重新编译内核4.2节ModuleNotFoundErrorPython包缺失或路径错误检查PYTHONPATH和环境变量usb_cam崩溃使用了错误的二进制文件使用OH交叉编译版本RKNN model file not found模型路径为相对路径YAML中使用绝对路径推理节点SIGSEGVLD_PRELOAD未设置确认完整环境变量6.1节输入维度错误输入顺序不匹配RKNN重排了输入state需放在最前面 关键文件与路径文件/路径作用位置models/502000/act_ros2_rknn.rknnRKNN模型文件主机端生成src/model_utils/model_utils/export_onnx_rknn.py模型导出脚本项目源码.agents/skills/rknn-convert/RKNN转换技能Agent技能库docs/RKNN_NPU_inference_on_BQ3588HM.md完整部署文档项目文档/data/ibrobot/install/板端安装目录BQ3588HM开发板 总结与下一步通过本指南您已成功在BQ3588HM开发板上部署了完整的IB-Robot系统实现了✅模型转换将ONNX模型转换为RKNN格式✅环境配置正确配置OpenHarmony ROS 2 RKNN环境✅内核编译支持USB ACM驱动和游戏手柄✅交叉编译部署第三方ROS包和Python依赖✅全链路启动实现摄像头→NPU→机械臂的完整闭环下一步建议尝试不同的视觉-动作策略模型优化推理性能探索量化压缩集成更多传感器雷达、IMU等开发自定义机器人应用IB-Robot框架为BQ3588HM开发板提供了强大的AI机器人开发能力让您能够快速部署和验证各种具身智能算法。祝您在机器人AI开发之路上取得更多成果提示更多详细信息请参考项目中的完整文档包括.agents/skills/目录下的各种技能说明和docs/目录下的技术文档。【免费下载链接】IB_RobotSave the code of IB-Robot, an AI robot execution framework developed by openEuler Embedded for embodied intelligence scenarios. It includes references to the forked version of tensormsg, references to lerobot fork, code references to the lerobot_ros2:ros2_ws branch, as well as some code related to development usability.项目地址: https://gitcode.com/openeuler/IB_Robot创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考