Difference between revisions of "NPU/zh"
From FriendlyELEC WiKi
(updated by API) |
(updated by API) |
||
(18 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | [[ | + | [[NPU|English]] |
− | == | + | ==Earlier version RKNPU2 SDK== |
− | === | + | Link to → [[NPU_SDK1.5.2/zh|v1.5.2]] |
+ | ==How to test NPU== | ||
+ | ===OS=== | ||
+ | Tested on the following OS: | ||
+ | ====Debian11 (bullseye)==== | ||
+ | * rk3588-sd-debian-bullseye-desktop-6.1-arm64-20241011.img.gz | ||
+ | ====Ubuntu20 (focal)==== | ||
+ | * rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20241011.img.gz | ||
+ | ===install rknpu=== | ||
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | cd ~ | |
− | + | export GIT_SSL_NO_VERIFY=1 | |
− | + | git clone https://github.com/airockchip/rknn-toolkit2.git | |
− | + | cd rknn-toolkit2/rknpu2 | |
− | + | sudo cp ./runtime/Linux/librknn_api/aarch64/* /usr/lib | |
+ | sudo cp ./runtime/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/ | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | === | + | ===check rknn version=== |
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | $ strings /usr/bin/rknn_server |grep 'build@' | |
− | + | 2.2.0 (3e10681 build@2024-09-06T10:26:49) | |
− | + | rknn_server version: 2.2.0 (3e10681 build@2024-09-06T10:26:49) | |
− | + | $ strings /usr/lib/librknnrt.so |grep 'librknnrt version:' | |
− | + | librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56) | |
</syntaxhighlight> | </syntaxhighlight> | ||
− | === | + | ===run rknn_yolov5_demo=== |
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | sudo apt-get update | |
− | + | sudo apt-get install -y gcc g++ make cmake | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
+ | # fix broken link | ||
+ | cd ~/rknn-toolkit2/rknpu2/examples/3rdparty/mpp/Linux/aarch64 | ||
+ | rm -f librockchip_mpp.so librockchip_mpp.so.1 | ||
+ | ln -s librockchip_mpp.so.0 librockchip_mpp.so | ||
+ | ln -s librockchip_mpp.so.0 librockchip_mpp.so.1 | ||
− | = | + | cd ~/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo |
− | + | chmod +x ./build-linux.sh | |
+ | sudo ln -s /usr/bin/gcc /usr/bin/aarch64-gcc | ||
+ | sudo ln -s /usr/bin/g++ /usr/bin/aarch64-g++ | ||
+ | export GCC_COMPILER=aarch64 | ||
+ | ./build-linux.sh -t rk3588 -a aarch64 -b Release | ||
+ | cd install/rknn_yolov5_demo_Linux | ||
+ | ./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg | ||
+ | </syntaxhighlight> | ||
+ | Transfer the generated out.jpg to PC to view the result: | ||
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | scp out.jpg xxx@YourIP:/tmp/ | |
</syntaxhighlight> | </syntaxhighlight> | ||
+ | [[File:Rknn_yolov5_demo_out.jpg|640px]] | ||
− | + | ===install rknn_toolkit on debian11=== | |
+ | ====install rknn_toolkit==== | ||
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | sudo apt-get update | |
− | + | sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip | |
− | + | cd ~/rknn-toolkit2 | |
− | + | pip3 install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.2.0-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/ | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | === | + | ====run python demo==== |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | $ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/ | |
− | + | $ python3 test.py | |
− | + | W rknn-toolkit-lite2 version: 2.2.0 | |
− | + | --> Load RKNN model | |
− | + | done | |
− | + | --> Init runtime environment | |
− | + | I RKNN: [06:50:39.456] RKNN Runtime Information, librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56) | |
− | + | I RKNN: [06:50:39.456] RKNN Driver Information, version: 0.9.3 | |
− | + | I RKNN: [06:50:39.456] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape | |
+ | W RKNN: [06:50:39.474] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes | ||
+ | W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.) | ||
+ | done | ||
+ | --> Running model | ||
+ | resnet18 | ||
+ | -----TOP 5----- | ||
+ | [812] score:0.999680 class:"space shuttle" | ||
+ | [404] score:0.000249 class:"airliner" | ||
+ | [657] score:0.000013 class:"missile" | ||
+ | [466] score:0.000009 class:"bullet train, bullet" | ||
+ | [895] score:0.000008 class:"warplane, military plane" | ||
+ | |||
done | done | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | == | + | ===install rknn_toolkit on ubuntu-focal=== |
− | + | ====build python3.9 from source==== | |
− | ==== | + | |
− | + | ||
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | sudo apt install build-essential libssl-dev libffi-dev software-properties-common \ | |
− | + | libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \ | |
− | + | tk-dev libgdbm-compat-dev libreadline-dev | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz | |
− | + | tar -xvf Python-3.9.16.tar.xz | |
− | + | cd Python-3.9.16/ | |
− | + | ./configure --enable-optimizations | |
− | + | make -j$(nproc) | |
− | + | sudo make install | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | + | ====install rknn_toolkit==== | |
− | + | <syntaxhighlight lang="text"> | |
− | <syntaxhighlight lang=" | + | python3.9 -m pip install --upgrade pip |
− | + | python3.9 -m pip install opencv-python | |
− | + | ||
− | + | cd ~ | |
− | + | [ -d rknn-toolkit2 ] || git clone https://github.com/airockchip/rknn-toolkit2.git \ | |
− | + | --depth 1 -b master | |
+ | cd rknn-toolkit2 | ||
+ | python3.9 -m pip install \ | ||
+ | ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.2.0-cp39-cp39-linux_aarch64.whl \ | ||
+ | -i https://pypi.tuna.tsinghua.edu.cn/simple/ | ||
+ | </syntaxhighlight> | ||
− | === | + | ====run python demo on ubuntu-focal==== |
− | + | ||
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/ | |
− | + | python3.9 test.py | |
</syntaxhighlight> | </syntaxhighlight> | ||
− | + | output: | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | W rknn-toolkit-lite2 version: 2.2.0 | |
− | + | --> Load RKNN model | |
− | + | done | |
− | + | --> Init runtime environment | |
− | + | I RKNN: [09:57:36.101] RKNN Runtime Information, librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56) | |
− | + | I RKNN: [09:57:36.101] RKNN Driver Information, version: 0.9.3 | |
− | + | I RKNN: [09:57:36.101] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape | |
− | + | W RKNN: [09:57:36.117] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes | |
− | + | W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.) | |
− | + | done | |
− | + | --> Running model | |
− | + | resnet18 | |
− | + | -----TOP 5----- | |
− | + | [812] score:0.999680 class:"space shuttle" | |
− | + | [404] score:0.000249 class:"airliner" | |
+ | [657] score:0.000013 class:"missile" | ||
+ | [466] score:0.000009 class:"bullet train, bullet" | ||
+ | [895] score:0.000008 class:"warplane, military plane" | ||
− | + | done | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | + | ==Doc== | |
− | === | + | https://github.com/rockchip-linux/rknpu2/tree/master/doc |
− | ==== | + | ==Other== |
− | + | ===查看NPU占有率=== | |
− | + | <syntaxhighlight lang="text"> | |
− | === | + | cat /sys/kernel/debug/rknpu/load |
− | ==== | + | </syntaxhighlight> |
+ | ===设置NPU频率=== | ||
+ | <syntaxhighlight lang="text"> | ||
+ | echo userspace > /sys/class/devfreq/fdab0000.npu/governor | ||
+ | echo 800000000 > /sys/class/devfreq/fdab0000.npu/min_freq | ||
+ | echo 1000000000 > /sys/class/devfreq/fdab0000.npu/max_freq | ||
+ | </syntaxhighlight> | ||
+ | ===查看NPU频率=== | ||
<syntaxhighlight lang="text"> | <syntaxhighlight lang="text"> | ||
− | + | cat /sys/class/devfreq/fdab0000.npu/cur_freq | |
− | + | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− |
Latest revision as of 10:00, 8 November 2024
Contents
1 Earlier version RKNPU2 SDK
Link to → v1.5.2
2 How to test NPU
2.1 OS
Tested on the following OS:
2.1.1 Debian11 (bullseye)
- rk3588-sd-debian-bullseye-desktop-6.1-arm64-20241011.img.gz
2.1.2 Ubuntu20 (focal)
- rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20241011.img.gz
2.2 install rknpu
cd ~ export GIT_SSL_NO_VERIFY=1 git clone https://github.com/airockchip/rknn-toolkit2.git cd rknn-toolkit2/rknpu2 sudo cp ./runtime/Linux/librknn_api/aarch64/* /usr/lib sudo cp ./runtime/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/
2.3 check rknn version
$ strings /usr/bin/rknn_server |grep 'build@' 2.2.0 (3e10681 build@2024-09-06T10:26:49) rknn_server version: 2.2.0 (3e10681 build@2024-09-06T10:26:49) $ strings /usr/lib/librknnrt.so |grep 'librknnrt version:' librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56)
2.4 run rknn_yolov5_demo
sudo apt-get update sudo apt-get install -y gcc g++ make cmake # fix broken link cd ~/rknn-toolkit2/rknpu2/examples/3rdparty/mpp/Linux/aarch64 rm -f librockchip_mpp.so librockchip_mpp.so.1 ln -s librockchip_mpp.so.0 librockchip_mpp.so ln -s librockchip_mpp.so.0 librockchip_mpp.so.1 cd ~/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo chmod +x ./build-linux.sh sudo ln -s /usr/bin/gcc /usr/bin/aarch64-gcc sudo ln -s /usr/bin/g++ /usr/bin/aarch64-g++ export GCC_COMPILER=aarch64 ./build-linux.sh -t rk3588 -a aarch64 -b Release cd install/rknn_yolov5_demo_Linux ./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg
Transfer the generated out.jpg to PC to view the result:
scp out.jpg xxx@YourIP:/tmp/
2.5 install rknn_toolkit on debian11
2.5.1 install rknn_toolkit
sudo apt-get update sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip cd ~/rknn-toolkit2 pip3 install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.2.0-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/
2.5.2 run python demo
$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/ $ python3 test.py W rknn-toolkit-lite2 version: 2.2.0 --> Load RKNN model done --> Init runtime environment I RKNN: [06:50:39.456] RKNN Runtime Information, librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56) I RKNN: [06:50:39.456] RKNN Driver Information, version: 0.9.3 I RKNN: [06:50:39.456] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape W RKNN: [06:50:39.474] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.) done --> Running model resnet18 -----TOP 5----- [812] score:0.999680 class:"space shuttle" [404] score:0.000249 class:"airliner" [657] score:0.000013 class:"missile" [466] score:0.000009 class:"bullet train, bullet" [895] score:0.000008 class:"warplane, military plane" done
2.6 install rknn_toolkit on ubuntu-focal
2.6.1 build python3.9 from source
sudo apt install build-essential libssl-dev libffi-dev software-properties-common \ libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \ tk-dev libgdbm-compat-dev libreadline-dev wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz tar -xvf Python-3.9.16.tar.xz cd Python-3.9.16/ ./configure --enable-optimizations make -j$(nproc) sudo make install
2.6.2 install rknn_toolkit
python3.9 -m pip install --upgrade pip python3.9 -m pip install opencv-python cd ~ [ -d rknn-toolkit2 ] || git clone https://github.com/airockchip/rknn-toolkit2.git \ --depth 1 -b master cd rknn-toolkit2 python3.9 -m pip install \ ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.2.0-cp39-cp39-linux_aarch64.whl \ -i https://pypi.tuna.tsinghua.edu.cn/simple/
2.6.3 run python demo on ubuntu-focal
cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/ python3.9 test.py
output:
W rknn-toolkit-lite2 version: 2.2.0 --> Load RKNN model done --> Init runtime environment I RKNN: [09:57:36.101] RKNN Runtime Information, librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56) I RKNN: [09:57:36.101] RKNN Driver Information, version: 0.9.3 I RKNN: [09:57:36.101] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape W RKNN: [09:57:36.117] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.) done --> Running model resnet18 -----TOP 5----- [812] score:0.999680 class:"space shuttle" [404] score:0.000249 class:"airliner" [657] score:0.000013 class:"missile" [466] score:0.000009 class:"bullet train, bullet" [895] score:0.000008 class:"warplane, military plane" done
3 Doc
https://github.com/rockchip-linux/rknpu2/tree/master/doc
4 Other
4.1 查看NPU占有率
cat /sys/kernel/debug/rknpu/load
4.2 设置NPU频率
echo userspace > /sys/class/devfreq/fdab0000.npu/governor echo 800000000 > /sys/class/devfreq/fdab0000.npu/min_freq echo 1000000000 > /sys/class/devfreq/fdab0000.npu/max_freq
4.3 查看NPU频率
cat /sys/class/devfreq/fdab0000.npu/cur_freq