Difference between revisions of "NPU"

From FriendlyELEC WiKi
Jump to: navigation, search
(updated by API)
 
(updated by API)
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
[[CPU/zh|查看中文]]
+
[[NPU/zh|查看中文]]
  
−
==RK3399==
+
==How to test NPU==
−
===查看cpu0-3 支持的频率===
+
===OS===
 +
Tested on the following OS:
 +
====Debian11 (bullseye)====
 +
* rk3588-sd-debian-bullseye-desktop-6.1-arm64-20240116.img.gz
 +
* rk3568-sd-debian-bullseye-desktop-6.1-arm64-20231113.img.gz
 +
====Ubuntu20 (focal)====
 +
* rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20240116.img.gz
 +
===install rknpu===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
+
export GIT_SSL_NO_VERIFY=1
 +
git clone https://github.com/rockchip-linux/rknpu2.git
 +
cd rknpu2
 +
git checkout tags/v1.5.2 -b v1.5.2
 +
sudo cp ./runtime/RK3588/Linux/librknn_api/aarch64/* /usr/lib
 +
sudo cp ./runtime/RK3588/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/
 
</syntaxhighlight>
 
</syntaxhighlight>
−
===锁定cpu0-3 的最高频率为816 mhz===
+
 
 +
===check rknn version===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
echo 816000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
+
strings /usr/bin/rknn_server |grep 'build@'
 +
strings /usr/lib/librknnrt.so |grep 'librknnrt version:'
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
===查看cpu4-5 支持的频率===
+
===run rknn_yolov5_demo===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
cat /sys/devices/system/cpu/cpu4/cpufreq/scaling_available_frequencies
+
sudo apt-get update
−
</syntaxhighlight>
+
sudo apt-get install -y gcc g++ make cmake
−
===锁定cpu4-5 的最高频率为1008 mhz===
+
cd examples/rknn_yolov5_demo
−
<syntaxhighlight lang="text">
+
./build-linux_RK3588.sh
−
echo 1008000 > /sys/devices/system/cpu/cpu4/cpufreq/scaling_max_freq
+
−
</syntaxhighlight>
+
  
−
===查看当前频率===
+
cd install/rknn_yolov5_demo_Linux
−
<syntaxhighlight lang="text">
+
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg
−
cat /sys/devices/system/cpu/cpu[04]/cpufreq/cpuinfo_cur_freq
+
 
</syntaxhighlight>
 
</syntaxhighlight>
−
===查看cpu4-5可用频率===
+
Transfer the generated out.jpg to PC to view the result:
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
/sys/devices/system/cpu/cpu4/cpufreq# cat scaling_available_frequencies
+
scp out.jpg xxx@YourIP:/tmp/
−
408000 600000 816000 1008000 1200000 1416000 1608000 1800000
+
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
[[File:Rknn_yolov5_demo_out.jpg|640px]]
  
−
==S5P4418/S5P6818==
+
===install rknn_toolkit on debian11===
−
===Setting CPU's Max Clock===
+
====install rknn_toolkit====
−
====Method 1====
+
−
The following command is used to temporarily limit the maximum CPU frequency to 1GHz:
+
−
echo 1000000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
+
−
 
+
−
====Method 2====
+
−
Modify the kernel source code:
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
arch/arm/plat-s5p4418/nanopi2/device.c
+
sudo apt-get update
 +
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
 +
cd ~
 +
git clone https://github.com/rockchip-linux/rknn-toolkit2.git
 +
(cd rknn-toolkit2 && git checkout tags/v1.5.2 -b v1.5.2)
 +
pip3 install ./rknn-toolkit2/rknn_toolkit_lite2/packages/rknn_toolkit_lite2-1.5.2-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
Delete the unwanted frequencies in the following array:
+
====run python demo====
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
static unsigned long dfs_freq_table[][2] = {
+
$ cd rknn-toolkit2/rknn_toolkit_lite2/examples/inference_with_lite/
−
    { 1400000, 1200000, },
+
$ python3 test.py
−
    { 1200000, 1120000, },
+
--> Load RKNN model
−
    { 1000000, 1040000, },
+
done
−
    {  800000, 1000000, },
+
--> Init runtime environment
−
    {  700000,  940000, },
+
I RKNN: [08:06:49.416] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
−
    {  600000,  940000, },
+
I RKNN: [08:06:49.416] RKNN Driver Information: version: 0.9.2
−
    {  500000,  940000, },
+
I RKNN: [08:06:49.416] RKNN Model Information: version: 6, toolkit version: 1.5.2-source_code(compiler version: 1.5.2 (71720f3fc@2023-08-
−
    {  400000,  940000, },
+
21T09:35:42)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_s
−
};
+
hape
−
</syntaxhighlight>
+
done
 +
--> Running model
 +
resnet18
 +
-----TOP 5-----
 +
[812]: 0.9996760487556458
 +
[404]: 0.00024927023332566023
 +
[657]: 1.449744013370946e-05
 +
[466 833]: 9.023910024552606e-06
 +
[466 833]: 9.023910024552606e-06
  
−
===CPU thermal===
 
−
Get CPU's working temperature by reading the following files:<br />
 
−
S5P4418/S5P6818/H3/H5/A64:<br />
 
−
/sys/class/thermal/thermal_zone0/temp<br />
 
−
RK3399, get CPU's working temperature and clock:<br />
 
−
<syntaxhighlight lang="text">
 
−
while true; do
 
−
t1=`cat /sys/class/thermal/thermal_zone0/temp`
 
−
t2=`cat /sys/class/thermal/thermal_zone1/temp`
 
−
f1=`cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq`
 
−
f2=`cat /sys/devices/system/cpu/cpu4/cpufreq/cpuinfo_cur_freq`
 
−
echo "$t1 $t2  $f1 $f2"
 
−
sleep 1
 
−
#cat /sys/class/thermal/thermal_zone[01]/temp
 
−
#cat /sys/devices/system/cpu/cpu[04]/cpufreq/cpuinfo_cur_freq
 
 
done
 
done
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
==H3==
+
===install rknn_toolkit on ubuntu===
−
===CPU DVFS===
+
====build python3.9 from source====
−
====Linux-4.14====
+
−
All H3 based boards have the same DVFS setting and the setting is in the "arch/arm/boot/dts/sun8i-h3-nanopi.dtsi" file:
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
&cpu0 {
+
sudo apt install build-essential libssl-dev libffi-dev software-properties-common \
−
operating-points = <
+
    libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \
−
1008000 1300000
+
    tk-dev libgdbm-compat-dev libreadline-dev
−
816000 1100000
+
−
624000 1100000
+
−
480000 1100000
+
−
312000 1100000
+
−
240000 1100000
+
−
120000 1100000
+
−
>;
+
−
#cooling-cells = <2>;
+
−
cooling-min-level = <0>;
+
−
cooling-max-level = <6>;
+
−
cpu0-supply = <&vdd_cpux>;
+
−
};
+
  
−
&cpu_thermal {
+
wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz
−
trips {
+
tar -xvf Python-3.9.16.tar.xz
−
cpu_warm: cpu_warm {
+
cd Python-3.9.16/
−
temperature = <65000>;
+
./configure --enable-optimizations
−
hysteresis = <2000>;
+
make -j$(nproc)
−
type = "passive";
+
sudo make install
−
};
+
−
cpu_hot: cpu_hot {
+
−
temperature = <75000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_very_hot: cpu_very_hot {
+
−
temperature = <90000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_crit: cpu_crit {
+
−
temperature = <105000>;
+
−
hysteresis = <2000>;
+
−
type = "critical";
+
−
};
+
−
};
+
−
 
+
−
cooling-maps {
+
−
cpu_warm_limit_cpu {
+
−
trip = <&cpu_warm>;
+
−
cooling-device = <&cpu0 THERMAL_NO_LIMIT 1>;
+
−
};
+
−
cpu_hot_limit_cpu {
+
−
trip = <&cpu_hot>;
+
−
cooling-device = <&cpu0 2 3>;
+
−
};
+
−
cpu_very_hot_limit_cpu {
+
−
trip = <&cpu_very_hot>;
+
−
cooling-device = <&cpu0 5 THERMAL_NO_LIMIT>;
+
−
};
+
−
};
+
−
};
+
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
===CPU ID===
+
====install rknn_toolkit====
−
====Linux-4.14====
+
−
{{FriendlyCoreAllwinner-CHIPID}}
+
−
 
+
−
===CPU thermal===
+
−
====Linux-4.14====
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
$ cat /sys/class/thermal/thermal_zone0/temp
+
pip install --upgrade pip
−
30489
+
pip install opencv-python
 +
cd ~
 +
git clone https://github.com/rockchip-linux/rknn-toolkit2.git
 +
(cd rknn-toolkit2 && git checkout tags/v1.5.2 -b v1.5.2)
 +
/usr/local/bin/python3.9 -m pip install ./rknn-toolkit2/rknn_toolkit_lite2/packages/rknn_toolkit_lite2-1.5.2-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/
 
</syntaxhighlight>
 
</syntaxhighlight>
−
The reading means the working temperature is 30.489 degrees Celsius.
 
  
−
==H5==
+
====run python demo====
−
===CPU DVFS===
+
−
====Linux-4.14====
+
−
For H5 based boards different boards have different DVFS settings.
+
−
Let's take the NanoPi NEO2 as an example the setting is in the "arch/arm64/boot/dts/allwinner/sun50i-h5-nanopi-neo2.dts" file:
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
&cpu0 {
+
$ cd rknn-toolkit2/rknn_toolkit_lite2/examples/inference_with_lite/
−
operating-points = <
+
$ python3 test.py
−
1008000 1100000
+
--> Load RKNN model
−
816000 1100000
+
done
−
624000 1100000
+
--> Init runtime environment
−
480000 1100000
+
I RKNN: [08:41:08.078] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
−
312000 1100000
+
I RKNN: [08:41:08.078] RKNN Driver Information: version: 0.9.2
−
240000 1100000
+
I RKNN: [08:41:08.080] RKNN Model Information: version: 6, toolkit version: 1.5.2-source_code(compiler version: 1.5.2 (71720f3fc@
−
120000 1100000
+
2023-08-21T09:35:42)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inferenc
−
>;
+
e type: static_shape
−
#cooling-cells = <2>;
+
done
−
cooling-min-level = <0>;
+
--> Running model
−
cooling-max-level = <6>;
+
resnet18
−
cpu0-supply = <&vdd_cpux>;
+
-----TOP 5-----
−
};
+
[812]: 0.9996760487556458
 +
[404]: 0.00024927023332566023
 +
[657]: 1.449744013370946e-05
 +
[466 833]: 9.023910024552606e-06
 +
[466 833]: 9.023910024552606e-06
  
−
&cpu_thermal {
+
done
−
trips {
+
−
cpu_warm: cpu_warm {
+
−
temperature = <65000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_hot: cpu_hot {
+
−
temperature = <75000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_very_hot: cpu_very_hot {
+
−
temperature = <90000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_crit: cpu_crit {
+
−
temperature = <105000>;
+
−
hysteresis = <2000>;
+
−
type = "critical";
+
−
};
+
−
};
+
−
 
+
−
cooling-maps {
+
−
cpu_warm_limit_cpu {
+
−
trip = <&cpu_warm>;
+
−
cooling-device = <&cpu0 THERMAL_NO_LIMIT 1>;
+
−
};
+
−
cpu_hot_limit_cpu {
+
−
trip = <&cpu_hot>;
+
−
cooling-device = <&cpu0 2 3>;
+
−
};
+
−
cpu_very_hot_limit_cpu {
+
−
trip = <&cpu_very_hot>;
+
−
cooling-device = <&cpu0 5 THERMAL_NO_LIMIT>;
+
−
};
+
−
};
+
−
};
+
 
</syntaxhighlight>
 
</syntaxhighlight>
−
 
+
==Doc==
−
===CPU ID===
+
https://github.com/rockchip-linux/rknpu2/tree/master/doc
−
====Linux-4.14====
+
==Other==
−
{{FriendlyCoreAllwinner-CHIPID}}
+
===View NPU Load===
−
 
+
<syntaxhighlight lang="text">
−
===CPU thermal===
+
cat /sys/kernel/debug/rknpu/load
−
====Linux-4.14====
+
</syntaxhighlight>
 +
===Set NPU freq===
 +
<syntaxhighlight lang="text">
 +
echo userspace > /sys/class/devfreq/fdab0000.npu/governor
 +
echo 800000000 > /sys/class/devfreq/fdab0000.npu/min_freq
 +
echo 1000000000 > /sys/class/devfreq/fdab0000.npu/max_freq
 +
</syntaxhighlight>
 +
===View NPU freq===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
$ cat /sys/class/thermal/thermal_zone0/temp
+
cat /sys/class/devfreq/fdab0000.npu/cur_freq
−
30489
+
 
</syntaxhighlight>
 
</syntaxhighlight>
−
The reading means the working temperature is 30.489 degrees Celsius.
 

Revision as of 05:54, 18 January 2024

查看中文

1 How to test NPU

1.1 OS

Tested on the following OS:

1.1.1 Debian11 (bullseye)

  • rk3588-sd-debian-bullseye-desktop-6.1-arm64-20240116.img.gz
  • rk3568-sd-debian-bullseye-desktop-6.1-arm64-20231113.img.gz

1.1.2 Ubuntu20 (focal)

  • rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20240116.img.gz

1.2 install rknpu

export GIT_SSL_NO_VERIFY=1
git clone https://github.com/rockchip-linux/rknpu2.git
cd rknpu2
git checkout tags/v1.5.2 -b v1.5.2
sudo cp ./runtime/RK3588/Linux/librknn_api/aarch64/* /usr/lib
sudo cp ./runtime/RK3588/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/

1.3 check rknn version

strings /usr/bin/rknn_server |grep 'build@'
strings /usr/lib/librknnrt.so |grep 'librknnrt version:'

1.4 run rknn_yolov5_demo

sudo apt-get update
sudo apt-get install -y gcc g++ make cmake
cd examples/rknn_yolov5_demo
./build-linux_RK3588.sh
 
cd install/rknn_yolov5_demo_Linux
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg

Transfer the generated out.jpg to PC to view the result: 

scp out.jpg xxx@YourIP:/tmp/

Rknn yolov5 demo out.jpg

1.5 install rknn_toolkit on debian11

1.5.1 install rknn_toolkit

sudo apt-get update
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
cd ~
git clone https://github.com/rockchip-linux/rknn-toolkit2.git
(cd rknn-toolkit2 && git checkout tags/v1.5.2 -b v1.5.2)
pip3 install ./rknn-toolkit2/rknn_toolkit_lite2/packages/rknn_toolkit_lite2-1.5.2-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/

1.5.2 run python demo

$ cd rknn-toolkit2/rknn_toolkit_lite2/examples/inference_with_lite/
$ python3 test.py
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [08:06:49.416] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
I RKNN: [08:06:49.416] RKNN Driver Information: version: 0.9.2
I RKNN: [08:06:49.416] RKNN Model Information: version: 6, toolkit version: 1.5.2-source_code(compiler version: 1.5.2 (71720f3fc@2023-08-
21T09:35:42)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_s
hape
done
--> Running model
resnet18
-----TOP 5-----
[812]: 0.9996760487556458
[404]: 0.00024927023332566023
[657]: 1.449744013370946e-05
[466 833]: 9.023910024552606e-06
[466 833]: 9.023910024552606e-06
 
done

1.6 install rknn_toolkit on ubuntu

1.6.1 build python3.9 from source

sudo apt install build-essential libssl-dev libffi-dev software-properties-common \
    libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \
    tk-dev libgdbm-compat-dev libreadline-dev
 
wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz
tar -xvf Python-3.9.16.tar.xz
cd Python-3.9.16/
./configure --enable-optimizations
make -j$(nproc)
sudo make install

1.6.2 install rknn_toolkit

pip install --upgrade pip
pip install opencv-python
cd ~
git clone https://github.com/rockchip-linux/rknn-toolkit2.git
(cd rknn-toolkit2 && git checkout tags/v1.5.2 -b v1.5.2)
/usr/local/bin/python3.9 -m pip install ./rknn-toolkit2/rknn_toolkit_lite2/packages/rknn_toolkit_lite2-1.5.2-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/

1.6.3 run python demo

$ cd rknn-toolkit2/rknn_toolkit_lite2/examples/inference_with_lite/
$ python3 test.py
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [08:41:08.078] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
I RKNN: [08:41:08.078] RKNN Driver Information: version: 0.9.2
I RKNN: [08:41:08.080] RKNN Model Information: version: 6, toolkit version: 1.5.2-source_code(compiler version: 1.5.2 (71720f3fc@
2023-08-21T09:35:42)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inferenc
e type: static_shape
done
--> Running model
resnet18
-----TOP 5-----
[812]: 0.9996760487556458
[404]: 0.00024927023332566023
[657]: 1.449744013370946e-05
[466 833]: 9.023910024552606e-06
[466 833]: 9.023910024552606e-06
 
done

2 Doc

https://github.com/rockchip-linux/rknpu2/tree/master/doc

3 Other

3.1 View NPU Load

cat /sys/kernel/debug/rknpu/load

3.2 Set NPU freq

echo userspace > /sys/class/devfreq/fdab0000.npu/governor
echo 800000000 > /sys/class/devfreq/fdab0000.npu/min_freq
echo 1000000000 > /sys/class/devfreq/fdab0000.npu/max_freq

3.3 View NPU freq

cat /sys/class/devfreq/fdab0000.npu/cur_freq
Retrieved from "https://wiki.friendlyelec.com/wiki/index.php?title=NPU&oldid=25569"