Difference between revisions of "NPU"

Latest revision as of 07:55, 11 May 2024

查看中文

1 Earlier version RKNPU2 SDK

Link to → v1.5.2

2 How to test NPU

2.1 OS

Tested on the following OS:

2.1.1 Debian11 (bullseye)

rk3588-sd-debian-bullseye-desktop-6.1-arm64-20240511.img.gz
rk3568-sd-debian-bullseye-desktop-6.1-arm64-20240511.img.gz

2.1.2 Ubuntu20 (focal)

rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20240511.img.gz

2.2 install rknpu

cd ~
export GIT_SSL_NO_VERIFY=1
git clone https://github.com/airockchip/rknn-toolkit2.git --depth 1 -b master
cd rknn-toolkit2/rknpu2
sudo cp ./runtime/Linux/librknn_api/aarch64/* /usr/lib
sudo cp ./runtime/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/
sudo cp ./runtime/Linux/librknn_api/include/* /usr/include/

2.3 check rknn version

$ strings /usr/bin/rknn_server |grep 'build@'
2.0.0b0 (18eacd0 build@2024-03-22T14:07:19)
rknn_server version: 2.0.0b0 (18eacd0 build@2024-03-22T14:07:19)
$ strings /usr/lib/librknnrt.so |grep 'librknnrt version:'
librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)

2.4 run rknn_yolov5_demo

sudo apt-get update
sudo apt-get install -y gcc g++ make cmake
 
# fix broken link
cd ~/rknn-toolkit2/rknpu2/examples/3rdparty/mpp/Linux/aarch64
rm -f librockchip_mpp.so librockchip_mpp.so.1
ln -s librockchip_mpp.so.0 librockchip_mpp.so
ln -s librockchip_mpp.so.0 librockchip_mpp.so.1
 
cd ~/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo
chmod +x ./build-linux.sh
sudo ln -s /usr/bin/gcc /usr/bin/aarch64-gcc
sudo ln -s /usr/bin/g++ /usr/bin/aarch64-g++
export GCC_COMPILER=aarch64
./build-linux.sh -t rk3588 -a aarch64 -b Release
cd install/rknn_yolov5_demo_Linux
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg

Transfer the generated out.jpg to PC to view the result:

scp out.jpg xxx@YourIP:/tmp/

2.5 install rknn_toolkit on debian11

2.5.1 install rknn_toolkit

sudo apt-get update
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
cd ~/rknn-toolkit2
pip3 install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.0.0b0-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/

2.5.2 run python demo

$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
$ python3 test.py
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [12:11:41.513] RKNN Runtime Information, librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
I RKNN: [12:11:41.514] RKNN Driver Information, version: 0.9.2
I RKNN: [12:11:41.514] RKNN Model Information, version: 6, toolkit version: 2.0.0b0+9bab5682(compiler version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11)), target: RKNPU lite, target platform: rk3566, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
done
--> Running model
resnet18
-----TOP 5-----
[812] score:0.999680 class:"space shuttle"
[404] score:0.000249 class:"airliner"
[657] score:0.000013 class:"missile"
[466] score:0.000009 class:"bullet train, bullet"
[895] score:0.000008 class:"warplane, military plane"
 
done

2.6 install rknn_toolkit on ubuntu

2.6.1 build python3.9 from source

sudo apt install build-essential libssl-dev libffi-dev software-properties-common \
    libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \
    tk-dev libgdbm-compat-dev libreadline-dev
 
wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz
tar -xvf Python-3.9.16.tar.xz
cd Python-3.9.16/
./configure --enable-optimizations
make -j$(nproc)
sudo make install

2.6.2 install rknn_toolkit

pip install --upgrade pip
pip install opencv-python
cd ~
git clone https://github.com/rockchip-linux/rknn-toolkit2.git
cd rknn-toolkit2
/usr/local/bin/python3.9 -m pip install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.0.0b0-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/

2.6.3 run python demo

$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
$ python3 test.py
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [12:11:41.513] RKNN Runtime Information, librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
I RKNN: [12:11:41.514] RKNN Driver Information, version: 0.9.2
I RKNN: [12:11:41.514] RKNN Model Information, version: 6, toolkit version: 2.0.0b0+9bab5682(compiler version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11)), target: RKNPU lite, target platform: rk3566, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
done
--> Running model
resnet18
-----TOP 5-----
[812] score:0.999680 class:"space shuttle"
[404] score:0.000249 class:"airliner"
[657] score:0.000013 class:"missile"
[466] score:0.000009 class:"bullet train, bullet"
[895] score:0.000008 class:"warplane, military plane"
 
done

3 Doc

https://github.com/rockchip-linux/rknpu2/tree/master/doc

4 Other

4.1 View NPU Load

cat /sys/kernel/debug/rknpu/load

4.2 Set NPU freq

echo userspace > /sys/class/devfreq/fdab0000.npu/governor
echo 800000000 > /sys/class/devfreq/fdab0000.npu/min_freq
echo 1000000000 > /sys/class/devfreq/fdab0000.npu/max_freq

4.3 View NPU freq

cat /sys/class/devfreq/fdab0000.npu/cur_freq

Difference between revisions of "NPU"

Latest revision as of 07:55, 11 May 2024

Contents

1 Earlier version RKNPU2 SDK

2 How to test NPU

2.1 OS

2.1.1 Debian11 (bullseye)

2.1.2 Ubuntu20 (focal)

2.2 install rknpu

2.3 check rknn version

2.4 run rknn_yolov5_demo

2.5 install rknn_toolkit on debian11

2.5.1 install rknn_toolkit

2.5.2 run python demo

2.6 install rknn_toolkit on ubuntu

2.6.1 build python3.9 from source

2.6.2 install rknn_toolkit

2.6.3 run python demo

3 Doc

4 Other

4.1 View NPU Load

4.2 Set NPU freq

4.3 View NPU freq

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Products

FAQ

Support

Tools

@@ Line 1: / Line 1: @@
-[[CPU/zh|查看中文]]
+[[NPU/zh|查看中文]]
-==RK3399==
+==Earlier version RKNPU2 SDK==
-===查看cpu0-3 支持的频率===
+Link to → [[NPU_SDK1.5.2|v1.5.2]]
+==How to test NPU==
+===OS===
+Tested on the following OS:
+====Debian11 (bullseye)====
+* rk3588-sd-debian-bullseye-desktop-6.1-arm64-20240511.img.gz
+* rk3568-sd-debian-bullseye-desktop-6.1-arm64-20240511.img.gz
+====Ubuntu20 (focal)====
+* rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20240511.img.gz
+===install rknpu===
 <syntaxhighlight lang="text">
-cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
+cd ~
-</syntaxhighlight>
+export GIT_SSL_NO_VERIFY=1
-===锁定cpu0-3 的最高频率为816 mhz===
+git clone https://github.com/airockchip/rknn-toolkit2.git --depth 1 -b master
-<syntaxhighlight lang="text">
+cd rknn-toolkit2/rknpu2
-echo 816000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
+sudo cp ./runtime/Linux/librknn_api/aarch64/* /usr/lib
+sudo cp ./runtime/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/
+sudo cp ./runtime/Linux/librknn_api/include/* /usr/include/
 </syntaxhighlight>
-===查看cpu4-5 支持的频率===
+===check rknn version===
-<syntaxhighlight lang="text">
-cat /sys/devices/system/cpu/cpu4/cpufreq/scaling_available_frequencies
-</syntaxhighlight>
-===锁定cpu4-5 的最高频率为1008 mhz===
 <syntaxhighlight lang="text">
-echo 1008000 > /sys/devices/system/cpu/cpu4/cpufreq/scaling_max_freq
+$ strings /usr/bin/rknn_server |grep 'build@'
+.0.0b0 (18eacd0 build@2024-03-22T14:07:19)
+rknn_server version: 2.0.0b0 (18eacd0 build@2024-03-22T14:07:19)
+$ strings /usr/lib/librknnrt.so |grep 'librknnrt version:'
+librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
 </syntaxhighlight>
-===查看当前频率===
+===run rknn_yolov5_demo===
 <syntaxhighlight lang="text">
-cat /sys/devices/system/cpu/cpu[04]/cpufreq/cpuinfo_cur_freq
+sudo apt-get update
-</syntaxhighlight>
+sudo apt-get install -y gcc g++ make cmake
-===查看cpu4-5可用频率===
-<syntaxhighlight lang="text">
-/sys/devices/system/cpu/cpu4/cpufreq# cat scaling_available_frequencies
-600000 816000 1008000 1200000 1416000 1608000 1800000
-</syntaxhighlight>
-==S5P4418/S5P6818==
+# fix broken link
-===Setting CPU's Max Clock===
+cd ~/rknn-toolkit2/rknpu2/examples/3rdparty/mpp/Linux/aarch64
-====Method 1====
+rm -f librockchip_mpp.so librockchip_mpp.so.1
-The following command is used to temporarily limit the maximum CPU frequency to 1GHz:
+ln -s librockchip_mpp.so.0 librockchip_mpp.so
-echo 1000000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
+ln -s librockchip_mpp.so.0 librockchip_mpp.so.1
-====Method 2====
+cd ~/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo
-Modify the kernel source code:
+chmod +x ./build-linux.sh
+sudo ln -s /usr/bin/gcc /usr/bin/aarch64-gcc
+sudo ln -s /usr/bin/g++ /usr/bin/aarch64-g++
+export GCC_COMPILER=aarch64
+./build-linux.sh -t rk3588 -a aarch64 -b Release
+cd install/rknn_yolov5_demo_Linux
+./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg
+</syntaxhighlight>
+Transfer the generated out.jpg to PC to view the result:
 <syntaxhighlight lang="text">
-arch/arm/plat-s5p4418/nanopi2/device.c
+scp out.jpg xxx@YourIP:/tmp/
 </syntaxhighlight>
+[[File:Rknn_yolov5_demo_out.jpg|640px]]
-Delete the unwanted frequencies in the following array:
+===install rknn_toolkit on debian11===
+====install rknn_toolkit====
 <syntaxhighlight lang="text">
-static unsigned long dfs_freq_table[][2] = {
+sudo apt-get update
-    { 1400000, 1200000, },
+sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
-    { 1200000, 1120000, },
+cd ~/rknn-toolkit2
-    { 1000000, 1040000, },
+pip3 install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.0.0b0-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/
-    {  800000, 1000000, },
-    {  700000,  940000, },
-    {  600000,  940000, },
-    {  500000,  940000, },
-    {  400000,  940000, },
-};
 </syntaxhighlight>
-===CPU thermal===
+====run python demo====
-Get CPU's working temperature by reading the following files:<br />
-S5P4418/S5P6818/H3/H5/A64:<br />
-/sys/class/thermal/thermal_zone0/temp<br />
-RK3399, get CPU's working temperature and clock:<br />
 <syntaxhighlight lang="text">
-while true; do
+$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
-	t1=`cat /sys/class/thermal/thermal_zone0/temp`
+$ python3 test.py
-	t2=`cat /sys/class/thermal/thermal_zone1/temp`
+--> Load RKNN model
-	f1=`cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq`
+done
-	f2=`cat /sys/devices/system/cpu/cpu4/cpufreq/cpuinfo_cur_freq`
+--> Init runtime environment
-	echo "$t1 $t2  $f1 $f2"
+I RKNN: [12:11:41.513] RKNN Runtime Information, librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
-	sleep 1
+I RKNN: [12:11:41.514] RKNN Driver Information, version: 0.9.2
-	#cat /sys/class/thermal/thermal_zone[01]/temp
+I RKNN: [12:11:41.514] RKNN Model Information, version: 6, toolkit version: 2.0.0b0+9bab5682(compiler version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11)), target: RKNPU lite, target platform: rk3566, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
-	#cat /sys/devices/system/cpu/cpu[04]/cpufreq/cpuinfo_cur_freq
+done
+--> Running model
+resnet18
+-----TOP 5-----
+[812] score:0.999680 class:"space shuttle"
+[404] score:0.000249 class:"airliner"
+[657] score:0.000013 class:"missile"
+[466] score:0.000009 class:"bullet train, bullet"
+[895] score:0.000008 class:"warplane, military plane"
 done
 </syntaxhighlight>
-==H3==
+===install rknn_toolkit on ubuntu===
-===CPU DVFS===
+====build python3.9 from source====
-====Linux-4.14====
-All H3 based boards have the same DVFS setting and the setting is in the "arch/arm/boot/dts/sun8i-h3-nanopi.dtsi" file:
 <syntaxhighlight lang="text">
-&cpu0 {
+sudo apt install build-essential libssl-dev libffi-dev software-properties-common \
-	operating-points = <
+    libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \
-		1008000	1300000
+    tk-dev libgdbm-compat-dev libreadline-dev
-	1100000
-	1100000
-	1100000
-	1100000
-	1100000
-	1100000
-		>;
-	#cooling-cells = <2>;
-	cooling-min-level = <0>;
-	cooling-max-level = <6>;
-	cpu0-supply = <&vdd_cpux>;
-};
-&cpu_thermal {
+wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz
-	trips {
+tar -xvf Python-3.9.16.tar.xz
-		cpu_warm: cpu_warm {
+cd Python-3.9.16/
-			temperature = <65000>;
+./configure --enable-optimizations
-			hysteresis = <2000>;
+make -j$(nproc)
-			type = "passive";
+sudo make install
-		};
-		cpu_hot: cpu_hot {
-			temperature = <75000>;
-			hysteresis = <2000>;
-			type = "passive";
-		};
-		cpu_very_hot: cpu_very_hot {
-			temperature = <90000>;
-			hysteresis = <2000>;
-			type = "passive";
-		};
-		cpu_crit: cpu_crit {
-			temperature = <105000>;
-			hysteresis = <2000>;
-			type = "critical";
-		};
-	};
-	cooling-maps {
-		cpu_warm_limit_cpu {
-			trip = <&cpu_warm>;
-			cooling-device = <&cpu0 THERMAL_NO_LIMIT 1>;
-		};
-		cpu_hot_limit_cpu {
-			trip = <&cpu_hot>;
-			cooling-device = <&cpu0 2 3>;
-		};
-		cpu_very_hot_limit_cpu {
-			trip = <&cpu_very_hot>;
-			cooling-device = <&cpu0 5 THERMAL_NO_LIMIT>;
-		};
-	};
-};
 </syntaxhighlight>
-===CPU ID===
+====install rknn_toolkit====
-====Linux-4.14====
-{{FriendlyCoreAllwinner-CHIPID}}
-===CPU thermal===
-====Linux-4.14====
 <syntaxhighlight lang="text">
-$ cat /sys/class/thermal/thermal_zone0/temp
+pip install --upgrade pip
+pip install opencv-python
+cd ~
+git clone https://github.com/rockchip-linux/rknn-toolkit2.git
+cd rknn-toolkit2
+/usr/local/bin/python3.9 -m pip install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.0.0b0-cp39-cp39-linux_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/
 </syntaxhighlight>
-The reading means the working temperature is 30.489 degrees Celsius.
-==H5==
+====run python demo====
-===CPU DVFS===
-====Linux-4.14====
-For H5 based boards different boards have different DVFS settings.
-Let's take the NanoPi NEO2 as an example the setting is in the "arch/arm64/boot/dts/allwinner/sun50i-h5-nanopi-neo2.dts" file:
 <syntaxhighlight lang="text">
-&cpu0 {
+$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
-	operating-points = <
+$ python3 test.py
-		1008000	1100000
+--> Load RKNN model
-	1100000
+done
-	1100000
+--> Init runtime environment
-	1100000
+I RKNN: [12:11:41.513] RKNN Runtime Information, librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
-	1100000
+I RKNN: [12:11:41.514] RKNN Driver Information, version: 0.9.2
-	1100000
+I RKNN: [12:11:41.514] RKNN Model Information, version: 6, toolkit version: 2.0.0b0+9bab5682(compiler version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11)), target: RKNPU lite, target platform: rk3566, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
-	1100000
+done
-		>;
+--> Running model
-	#cooling-cells = <2>;
+resnet18
-	cooling-min-level = <0>;
+-----TOP 5-----
-	cooling-max-level = <6>;
+[812] score:0.999680 class:"space shuttle"
-	cpu0-supply = <&vdd_cpux>;
+[404] score:0.000249 class:"airliner"
-};
+[657] score:0.000013 class:"missile"
+[466] score:0.000009 class:"bullet train, bullet"
+[895] score:0.000008 class:"warplane, military plane"
-&cpu_thermal {
+done
-	trips {
-		cpu_warm: cpu_warm {
-			temperature = <65000>;
-			hysteresis = <2000>;
-			type = "passive";
-		};
-		cpu_hot: cpu_hot {
-			temperature = <75000>;
-			hysteresis = <2000>;
-			type = "passive";
-		};
-		cpu_very_hot: cpu_very_hot {
-			temperature = <90000>;
-			hysteresis = <2000>;
-			type = "passive";
-		};
-		cpu_crit: cpu_crit {
-			temperature = <105000>;
-			hysteresis = <2000>;
-			type = "critical";
-		};
-	};
-	cooling-maps {
-		cpu_warm_limit_cpu {
-			trip = <&cpu_warm>;
-			cooling-device = <&cpu0 THERMAL_NO_LIMIT 1>;
-		};
-		cpu_hot_limit_cpu {
-			trip = <&cpu_hot>;
-			cooling-device = <&cpu0 2 3>;
-		};
-		cpu_very_hot_limit_cpu {
-			trip = <&cpu_very_hot>;
-			cooling-device = <&cpu0 5 THERMAL_NO_LIMIT>;
-		};
-	};
-};
 </syntaxhighlight>
+==Doc==
-===CPU ID===
+https://github.com/rockchip-linux/rknpu2/tree/master/doc
-====Linux-4.14====
+==Other==
-{{FriendlyCoreAllwinner-CHIPID}}
+===View NPU Load===
+<syntaxhighlight lang="text">
-===CPU thermal===
+cat /sys/kernel/debug/rknpu/load
-====Linux-4.14====
+</syntaxhighlight>
+===Set NPU freq===
+<syntaxhighlight lang="text">
+echo userspace > /sys/class/devfreq/fdab0000.npu/governor
+echo 800000000 > /sys/class/devfreq/fdab0000.npu/min_freq
+echo 1000000000 > /sys/class/devfreq/fdab0000.npu/max_freq
+</syntaxhighlight>
+===View NPU freq===
 <syntaxhighlight lang="text">
-$ cat /sys/class/thermal/thermal_zone0/temp
+cat /sys/class/devfreq/fdab0000.npu/cur_freq
 </syntaxhighlight>
-The reading means the working temperature is 30.489 degrees Celsius.