Difference between revisions of "NPU/zh"

From FriendlyELEC WiKi
Jump to: navigation, search
(updated by API)
 
(updated by API)
 
(18 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[CPU|English]]
+
[[NPU|English]]
  
==RK3399==
+
==Earlier version RKNPU2 SDK==
===查看cpu0-3 支持的频率===
+
Link to → [[NPU_SDK1.5.2/zh|v1.5.2]]
 +
==How to test NPU==
 +
===OS===
 +
Tested on the following OS:
 +
====Debian11 (bullseye)====
 +
* rk3588-sd-debian-bullseye-desktop-6.1-arm64-20241011.img.gz
 +
====Ubuntu20 (focal)====
 +
* rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20241011.img.gz
 +
===install rknpu===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
+
cd ~
</syntaxhighlight>
+
export GIT_SSL_NO_VERIFY=1
===锁定cpu0-3 的最高频率为816 mhz===
+
git clone https://github.com/airockchip/rknn-toolkit2.git
<syntaxhighlight lang="text">
+
cd rknn-toolkit2/rknpu2
echo 816000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
+
sudo cp ./runtime/Linux/librknn_api/aarch64/* /usr/lib
 +
sudo cp ./runtime/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===查看cpu4-5 支持的频率===
+
===check rknn version===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
cat /sys/devices/system/cpu/cpu4/cpufreq/scaling_available_frequencies
+
$ strings /usr/bin/rknn_server |grep 'build@'
</syntaxhighlight>
+
2.2.0 (3e10681 build@2024-09-06T10:26:49)
===锁定cpu4-5 的最高频率为1008 mhz===
+
rknn_server version: 2.2.0 (3e10681 build@2024-09-06T10:26:49)
<syntaxhighlight lang="text">
+
$ strings /usr/lib/librknnrt.so |grep 'librknnrt version:'
echo 1008000 > /sys/devices/system/cpu/cpu4/cpufreq/scaling_max_freq
+
librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56)
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===查看当前频率===
+
===run rknn_yolov5_demo===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
cat /sys/devices/system/cpu/cpu[04]/cpufreq/cpuinfo_cur_freq
+
sudo apt-get update
</syntaxhighlight>
+
sudo apt-get install -y gcc g++ make cmake
==S5P4418/S5P6818==
+
===锁定CPU的最高频率===
+
====Method 1====
+
The following command is used to temporarily limit the maximum CPU frequency to 1GHz:
+
echo 1000000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
+
  
 +
# fix broken link
 +
cd ~/rknn-toolkit2/rknpu2/examples/3rdparty/mpp/Linux/aarch64
 +
rm -f librockchip_mpp.so librockchip_mpp.so.1
 +
ln -s librockchip_mpp.so.0 librockchip_mpp.so
 +
ln -s librockchip_mpp.so.0 librockchip_mpp.so.1
  
====Method 2====
+
cd ~/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo
Modify the kernel source code:
+
chmod +x ./build-linux.sh
 +
sudo ln -s /usr/bin/gcc /usr/bin/aarch64-gcc
 +
sudo ln -s /usr/bin/g++ /usr/bin/aarch64-g++
 +
export GCC_COMPILER=aarch64
 +
./build-linux.sh -t rk3588 -a aarch64 -b Release
 +
cd install/rknn_yolov5_demo_Linux
 +
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg
 +
</syntaxhighlight>
 +
Transfer the generated out.jpg to PC to view the result:
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
arch/arm/plat-s5p4418/nanopi2/device.c
+
scp out.jpg xxx@YourIP:/tmp/
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
[[File:Rknn_yolov5_demo_out.jpg|640px]]
  
Delete the unwanted frequencies in the following array:
+
===install rknn_toolkit on debian11===
 +
====install rknn_toolkit====
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
static unsigned long dfs_freq_table[][2] = {
+
sudo apt-get update
    { 1400000, 1200000, },
+
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
    { 1200000, 1120000, },
+
cd ~/rknn-toolkit2
    { 1000000, 1040000, },
+
pip3 install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.2.0-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/
    {  800000, 1000000, },
+
    {  700000,  940000, },
+
    {  600000,  940000, },
+
    {  500000,  940000, },
+
    {  400000,  940000, },
+
};
+
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===CPU thermal===
+
====run python demo====
读取以下文件获得当前的CPU温度:<br />
+
S5P4418/S5P6818/H3/H5:<br />
+
/sys/class/thermal/thermal_zone0/temp<br />
+
RK3399,获取CPU温度和运行频率:<br />
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
while true; do
+
$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
t1=`cat /sys/class/thermal/thermal_zone0/temp`
+
$ python3 test.py
t2=`cat /sys/class/thermal/thermal_zone1/temp`
+
W rknn-toolkit-lite2 version: 2.2.0
f1=`cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq`
+
--> Load RKNN model
f2=`cat /sys/devices/system/cpu/cpu4/cpufreq/cpuinfo_cur_freq`
+
done
echo "$t1 $t2  $f1 $f2"
+
--> Init runtime environment
sleep 1
+
I RKNN: [06:50:39.456] RKNN Runtime Information, librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56)
#cat /sys/class/thermal/thermal_zone[01]/temp
+
I RKNN: [06:50:39.456] RKNN Driver Information, version: 0.9.3
#cat /sys/devices/system/cpu/cpu[04]/cpufreq/cpuinfo_cur_freq
+
I RKNN: [06:50:39.456] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
 +
W RKNN: [06:50:39.474] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
 +
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
 +
done
 +
--> Running model
 +
resnet18
 +
-----TOP 5-----
 +
[812] score:0.999680 class:"space shuttle"
 +
[404] score:0.000249 class:"airliner"
 +
[657] score:0.000013 class:"missile"
 +
[466] score:0.000009 class:"bullet train, bullet"
 +
[895] score:0.000008 class:"warplane, military plane"
 +
 
 
done
 
done
 
</syntaxhighlight>
 
</syntaxhighlight>
  
==H3==
+
===install rknn_toolkit on ubuntu-focal===
===CPU DVFS===
+
====build python3.9 from source====
====Linux-4.14====
+
对于H3系列的开发板,使用的都是同样的DVFS设置,相关代码位于arch/arm/boot/dts/sun8i-h3-nanopi.dtsi:
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
&cpu0 {
+
sudo apt install build-essential libssl-dev libffi-dev software-properties-common \
operating-points = <
+
    libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \
1008000 1300000
+
    tk-dev libgdbm-compat-dev libreadline-dev
816000 1100000
+
624000 1100000
+
480000 1100000
+
312000 1100000
+
240000 1100000
+
120000 1100000
+
>;
+
#cooling-cells = <2>;
+
cooling-min-level = <0>;
+
cooling-max-level = <6>;
+
cpu0-supply = <&vdd_cpux>;
+
};
+
 
+
&cpu_thermal {
+
trips {
+
cpu_warm: cpu_warm {
+
temperature = <65000>;
+
hysteresis = <2000>;
+
type = "passive";
+
};
+
cpu_hot: cpu_hot {
+
temperature = <75000>;
+
hysteresis = <2000>;
+
type = "passive";
+
};
+
cpu_very_hot: cpu_very_hot {
+
temperature = <90000>;
+
hysteresis = <2000>;
+
type = "passive";
+
};
+
cpu_crit: cpu_crit {
+
temperature = <105000>;
+
hysteresis = <2000>;
+
type = "critical";
+
};
+
};
+
  
cooling-maps {
+
wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz
cpu_warm_limit_cpu {
+
tar -xvf Python-3.9.16.tar.xz
trip = <&cpu_warm>;
+
cd Python-3.9.16/
cooling-device = <&cpu0 THERMAL_NO_LIMIT 1>;
+
./configure --enable-optimizations
};
+
make -j$(nproc)
cpu_hot_limit_cpu {
+
sudo make install
trip = <&cpu_hot>;
+
cooling-device = <&cpu0 2 3>;
+
};
+
cpu_very_hot_limit_cpu {
+
trip = <&cpu_very_hot>;
+
cooling-device = <&cpu0 5 THERMAL_NO_LIMIT>;
+
};
+
};
+
};
+
 
</syntaxhighlight>
 
</syntaxhighlight>
  
* 关闭cpu核
+
====install rknn_toolkit====
以关闭cpu1为例:
+
<syntaxhighlight lang="text">
<syntaxhighlight lang="bash">
+
python3.9 -m pip install --upgrade pip
echo 0 > /sys/devices/system/cpu/cpu1/online
+
python3.9 -m pip install opencv-python
</syntaxhighlight>
+
  
===CPU ID===
+
cd ~
====Linux-4.14====
+
[ -d rknn-toolkit2 ] || git clone https://github.com/airockchip/rknn-toolkit2.git \
{{FriendlyCoreAllwinner-CHIPID/zh}}
+
    --depth 1 -b master
 +
cd rknn-toolkit2
 +
python3.9 -m pip install \
 +
    ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.2.0-cp39-cp39-linux_aarch64.whl \
 +
    -i https://pypi.tuna.tsinghua.edu.cn/simple/
 +
</syntaxhighlight>
  
===CPU thermal===
+
====run python demo on ubuntu-focal====
====Linux-4.14====
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
$ cat /sys/class/thermal/thermal_zone0/temp
+
cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
30489
+
python3.9 test.py
 
</syntaxhighlight>
 
</syntaxhighlight>
表示当前CPU温度为30.489摄氏度。
+
output:
 
+
==H5==
+
===CPU DVFS===
+
====Linux-4.14====
+
对于H5系列的开发板,不同型号的DVFS设置是不一样的。
+
以NanoPi NEO2为例,相关代码位于arch/arm64/boot/dts/allwinner/sun50i-h5-nanopi-neo2.dts:
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
&cpu0 {
+
W rknn-toolkit-lite2 version: 2.2.0
operating-points = <
+
--> Load RKNN model
1008000 1100000
+
done
816000 1100000
+
--> Init runtime environment
624000 1100000
+
I RKNN: [09:57:36.101] RKNN Runtime Information, librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56)
480000 1100000
+
I RKNN: [09:57:36.101] RKNN Driver Information, version: 0.9.3
312000 1100000
+
I RKNN: [09:57:36.101] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
240000 1100000
+
W RKNN: [09:57:36.117] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
120000 1100000
+
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
>;
+
done
#cooling-cells = <2>;
+
--> Running model
cooling-min-level = <0>;
+
resnet18
cooling-max-level = <6>;
+
-----TOP 5-----
cpu0-supply = <&vdd_cpux>;
+
[812] score:0.999680 class:"space shuttle"
};
+
[404] score:0.000249 class:"airliner"
 +
[657] score:0.000013 class:"missile"
 +
[466] score:0.000009 class:"bullet train, bullet"
 +
[895] score:0.000008 class:"warplane, military plane"
  
&cpu_thermal {
+
done
trips {
+
cpu_warm: cpu_warm {
+
temperature = <65000>;
+
hysteresis = <2000>;
+
type = "passive";
+
};
+
cpu_hot: cpu_hot {
+
temperature = <75000>;
+
hysteresis = <2000>;
+
type = "passive";
+
};
+
cpu_very_hot: cpu_very_hot {
+
temperature = <90000>;
+
hysteresis = <2000>;
+
type = "passive";
+
};
+
cpu_crit: cpu_crit {
+
temperature = <105000>;
+
hysteresis = <2000>;
+
type = "critical";
+
};
+
};
+
 
+
cooling-maps {
+
cpu_warm_limit_cpu {
+
trip = <&cpu_warm>;
+
cooling-device = <&cpu0 THERMAL_NO_LIMIT 1>;
+
};
+
cpu_hot_limit_cpu {
+
trip = <&cpu_hot>;
+
cooling-device = <&cpu0 2 3>;
+
};
+
cpu_very_hot_limit_cpu {
+
trip = <&cpu_very_hot>;
+
cooling-device = <&cpu0 5 THERMAL_NO_LIMIT>;
+
};
+
};
+
};
+
 
</syntaxhighlight>
 
</syntaxhighlight>
 
+
==Doc==
===CPU ID===
+
https://github.com/rockchip-linux/rknpu2/tree/master/doc
====Linux-4.14====
+
==Other==
{{FriendlyCoreAllwinner-CHIPID/zh}}
+
===查看NPU占有率===
 
+
<syntaxhighlight lang="text">
===CPU thermal===
+
cat /sys/kernel/debug/rknpu/load
====Linux-4.14====
+
</syntaxhighlight>
 +
===设置NPU频率===
 +
<syntaxhighlight lang="text">
 +
echo userspace > /sys/class/devfreq/fdab0000.npu/governor
 +
echo 800000000 > /sys/class/devfreq/fdab0000.npu/min_freq
 +
echo 1000000000 > /sys/class/devfreq/fdab0000.npu/max_freq
 +
</syntaxhighlight>
 +
===查看NPU频率===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
$ cat /sys/class/thermal/thermal_zone0/temp
+
cat /sys/class/devfreq/fdab0000.npu/cur_freq
30489
+
 
</syntaxhighlight>
 
</syntaxhighlight>
表示当前CPU温度为30.489摄氏度。
 

Latest revision as of 10:00, 8 November 2024

English

1 Earlier version RKNPU2 SDK

Link to → v1.5.2

2 How to test NPU

2.1 OS

Tested on the following OS:

2.1.1 Debian11 (bullseye)

  • rk3588-sd-debian-bullseye-desktop-6.1-arm64-20241011.img.gz

2.1.2 Ubuntu20 (focal)

  • rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20241011.img.gz

2.2 install rknpu

cd ~
export GIT_SSL_NO_VERIFY=1
git clone https://github.com/airockchip/rknn-toolkit2.git
cd rknn-toolkit2/rknpu2
sudo cp ./runtime/Linux/librknn_api/aarch64/* /usr/lib
sudo cp ./runtime/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/

2.3 check rknn version

$ strings /usr/bin/rknn_server |grep 'build@'
2.2.0 (3e10681 build@2024-09-06T10:26:49)
rknn_server version: 2.2.0 (3e10681 build@2024-09-06T10:26:49)
$ strings /usr/lib/librknnrt.so |grep 'librknnrt version:'
librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56)

2.4 run rknn_yolov5_demo

sudo apt-get update
sudo apt-get install -y gcc g++ make cmake
 
# fix broken link
cd ~/rknn-toolkit2/rknpu2/examples/3rdparty/mpp/Linux/aarch64
rm -f librockchip_mpp.so librockchip_mpp.so.1
ln -s librockchip_mpp.so.0 librockchip_mpp.so
ln -s librockchip_mpp.so.0 librockchip_mpp.so.1
 
cd ~/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo
chmod +x ./build-linux.sh
sudo ln -s /usr/bin/gcc /usr/bin/aarch64-gcc
sudo ln -s /usr/bin/g++ /usr/bin/aarch64-g++
export GCC_COMPILER=aarch64
./build-linux.sh -t rk3588 -a aarch64 -b Release
cd install/rknn_yolov5_demo_Linux
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg

Transfer the generated out.jpg to PC to view the result:

scp out.jpg xxx@YourIP:/tmp/

Rknn yolov5 demo out.jpg

2.5 install rknn_toolkit on debian11

2.5.1 install rknn_toolkit

sudo apt-get update
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
cd ~/rknn-toolkit2
pip3 install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.2.0-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/

2.5.2 run python demo

$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
$ python3 test.py
W rknn-toolkit-lite2 version: 2.2.0
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [06:50:39.456] RKNN Runtime Information, librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56)
I RKNN: [06:50:39.456] RKNN Driver Information, version: 0.9.3
I RKNN: [06:50:39.456] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
W RKNN: [06:50:39.474] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
done
--> Running model
resnet18
-----TOP 5-----
[812] score:0.999680 class:"space shuttle"
[404] score:0.000249 class:"airliner"
[657] score:0.000013 class:"missile"
[466] score:0.000009 class:"bullet train, bullet"
[895] score:0.000008 class:"warplane, military plane"
 
done

2.6 install rknn_toolkit on ubuntu-focal

2.6.1 build python3.9 from source

sudo apt install build-essential libssl-dev libffi-dev software-properties-common \
    libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \
    tk-dev libgdbm-compat-dev libreadline-dev
 
wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz
tar -xvf Python-3.9.16.tar.xz
cd Python-3.9.16/
./configure --enable-optimizations
make -j$(nproc)
sudo make install

2.6.2 install rknn_toolkit

python3.9 -m pip install --upgrade pip
python3.9 -m pip install opencv-python
 
cd ~
[ -d rknn-toolkit2 ] || git clone https://github.com/airockchip/rknn-toolkit2.git \
    --depth 1 -b master
cd rknn-toolkit2
python3.9 -m pip install \
    ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.2.0-cp39-cp39-linux_aarch64.whl \
    -i https://pypi.tuna.tsinghua.edu.cn/simple/

2.6.3 run python demo on ubuntu-focal

cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
python3.9 test.py

output:

W rknn-toolkit-lite2 version: 2.2.0
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [09:57:36.101] RKNN Runtime Information, librknnrt version: 2.2.0 (c195366594@2024-09-14T12:18:56)
I RKNN: [09:57:36.101] RKNN Driver Information, version: 0.9.3
I RKNN: [09:57:36.101] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
W RKNN: [09:57:36.117] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
done
--> Running model
resnet18
-----TOP 5-----
[812] score:0.999680 class:"space shuttle"
[404] score:0.000249 class:"airliner"
[657] score:0.000013 class:"missile"
[466] score:0.000009 class:"bullet train, bullet"
[895] score:0.000008 class:"warplane, military plane"
 
done

3 Doc

https://github.com/rockchip-linux/rknpu2/tree/master/doc

4 Other

4.1 查看NPU占有率

cat /sys/kernel/debug/rknpu/load

4.2 设置NPU频率

echo userspace > /sys/class/devfreq/fdab0000.npu/governor
echo 800000000 > /sys/class/devfreq/fdab0000.npu/min_freq
echo 1000000000 > /sys/class/devfreq/fdab0000.npu/max_freq

4.3 查看NPU频率

cat /sys/class/devfreq/fdab0000.npu/cur_freq