Difference between revisions of "NPU"

From FriendlyELEC WiKi
Jump to: navigation, search
(updated by API)
 
(updated by API)
 
(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
[[CPU/zh|查看中文]]
+
[[NPU/zh|查看中文]]
  
−
==RK3399==
+
==Earlier version RKNPU2 SDK==
−
===查看cpu0-3 支持的频率===
+
Link to → [[NPU_SDK1.5.2|v1.5.2]]
 +
==Check NPU driver version==
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
+
$ sudo cat /sys/kernel/debug/rknpu/version
 +
RKNPU driver: v0.9.8
 
</syntaxhighlight>
 
</syntaxhighlight>
−
===锁定cpu0-3 的最高频率为816 mhz===
+
==RKNN Quick Example==
 +
===OS===
 +
Tested on the following OS:
 +
====Debian11 (bullseye)====
 +
* rk3588-sd-debian-bullseye-desktop-6.1-arm64-20250117.img.gz
 +
====Ubuntu20 (focal)====
 +
* rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20250117.img.gz
 +
===Installing RKNN Runtime===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
echo 816000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
+
cd ~
 +
export GIT_SSL_NO_VERIFY=1
 +
git clone https://github.com/airockchip/rknn-toolkit2.git
 +
cd rknn-toolkit2/rknpu2
 +
sudo cp ./runtime/Linux/librknn_api/aarch64/* /usr/lib
 +
sudo cp ./runtime/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/
 +
sudo cp ./runtime/Linux/librknn_api/include/* /usr/include/
 +
</syntaxhighlight>
 +
===Check rknn version===
 +
<syntaxhighlight lang="text">
 +
$ strings /usr/bin/rknn_server |grep 'build@'
 +
2.3.0 (e80ac5c build@2024-11-07T12:52:53)
 +
rknn_server version: 2.3.0 (e80ac5c build@2024-11-07T12:52:53)
 +
$ strings /usr/lib/librknnrt.so |grep 'librknnrt version:'
 +
librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
===查看cpu4-5 支持的频率===
+
===C++ Example: Build and run the YOLOv5 demo===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
cat /sys/devices/system/cpu/cpu4/cpufreq/scaling_available_frequencies
+
sudo apt-get update
 +
sudo apt-get install -y gcc g++ make cmake
 +
 
 +
# fix broken link
 +
cd ~/rknn-toolkit2/rknpu2/examples/3rdparty/mpp/Linux/aarch64
 +
rm -f librockchip_mpp.so librockchip_mpp.so.1
 +
ln -s librockchip_mpp.so.0 librockchip_mpp.so
 +
ln -s librockchip_mpp.so.0 librockchip_mpp.so.1
 +
 
 +
cd ~/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo
 +
chmod +x ./build-linux.sh
 +
sudo ln -s /usr/bin/gcc /usr/bin/aarch64-gcc
 +
sudo ln -s /usr/bin/g++ /usr/bin/aarch64-g++
 +
export GCC_COMPILER=aarch64
 +
./build-linux.sh -t rk3588 -a aarch64 -b Release
 +
cd install/rknn_yolov5_demo_Linux
 +
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg
 
</syntaxhighlight>
 
</syntaxhighlight>
−
===锁定cpu4-5 的最高频率为1008 mhz===
+
Transfer the generated out.jpg to PC to view the result:
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
echo 1008000 > /sys/devices/system/cpu/cpu4/cpufreq/scaling_max_freq
+
scp out.jpg xxx@YourIP:/tmp/
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
[[File:Rknn_yolov5_demo_out.jpg|640px]]
  
−
===查看当前频率===
+
===Python Example: Install and Test RKNN Toolkit Lite2 on Debian11===
 +
====Install RKNN Toolkit Lite2====
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
cat /sys/devices/system/cpu/cpu[04]/cpufreq/cpuinfo_cur_freq
+
sudo apt-get update
 +
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
 +
cd ~/rknn-toolkit2
 +
pip3 install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.3.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/
 
</syntaxhighlight>
 
</syntaxhighlight>
−
===查看cpu4-5可用频率===
+
 
 +
====Run the python example====
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
/sys/devices/system/cpu/cpu4/cpufreq# cat scaling_available_frequencies
+
$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
−
408000 600000 816000 1008000 1200000 1416000 1608000 1800000
+
$ python3 test.py
 +
W rknn-toolkit-lite2 version: 2.3.0
 +
--> Load RKNN model
 +
done
 +
--> Init runtime environment
 +
I RKNN: [06:29:43.602] RKNN Runtime Information, librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
 +
I RKNN: [06:29:43.602] RKNN Driver Information, version: 0.9.8
 +
I RKNN: [06:29:43.602] RKNN Model Information, version: 6, toolkit version: 2.3.0(compiler version: 2.3.0 (c949ad889d@2024-11-07T11:39:30)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
 +
W RKNN: [06:29:43.617] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
 +
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
 +
done
 +
--> Running model
 +
resnet18
 +
-----TOP 5-----
 +
[812] score:0.999680 class:"space shuttle"
 +
[404] score:0.000249 class:"airliner"
 +
[657] score:0.000013 class:"missile"
 +
[466] score:0.000009 class:"bullet train, bullet"
 +
[895] score:0.000008 class:"warplane, military plane"
 +
 
 +
done
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
==S5P4418/S5P6818==
+
===Python Example: Install and Test RKNN Toolkit Lite2 on Ubuntu 20.04 (Focal)===
−
===Setting CPU's Max Clock===
+
====Compile and install python3.9====
−
====Method 1====
+
−
The following command is used to temporarily limit the maximum CPU frequency to 1GHz:
+
−
echo 1000000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
+
−
 
+
−
====Method 2====
+
−
Modify the kernel source code:
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
arch/arm/plat-s5p4418/nanopi2/device.c
+
sudo apt install build-essential libssl-dev libffi-dev software-properties-common \
 +
    libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \
 +
    tk-dev libgdbm-compat-dev libreadline-dev
 +
 
 +
wget https://www.python.org/ftp/python/3.9.21/Python-3.9.21.tar.xz
 +
tar -xvf Python-3.9.21.tar.xz
 +
cd Python-3.9.21/
 +
./configure --enable-optimizations
 +
make -j$(nproc)
 +
sudo make install
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
Delete the unwanted frequencies in the following array:
+
====Install RKNN Toolkit Lite2====
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
static unsigned long dfs_freq_table[][2] = {
+
python3.9 -m pip install --upgrade pip
−
    { 1400000, 1200000, },
+
python3.9 -m pip install opencv-python
−
    { 1200000, 1120000, },
+
 
−
    { 1000000, 1040000, },
+
cd ~
−
    {  800000, 1000000, },
+
[ -d rknn-toolkit2 ] || git clone https://github.com/airockchip/rknn-toolkit2.git \
−
     {  700000,  940000, },
+
     --depth 1 -b master
−
    {  600000,  940000, },
+
cd rknn-toolkit2
−
     {  500000,  940000, },
+
python3.9 -m pip install \
−
     {  400000,  940000, },
+
     ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.3.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl \
−
};
+
     -i https://pypi.tuna.tsinghua.edu.cn/simple/
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
===CPU thermal===
+
====Run the python example====
−
Get CPU's working temperature by reading the following files:<br />
+
−
S5P4418/S5P6818/H3/H5/A64:<br />
+
−
/sys/class/thermal/thermal_zone0/temp<br />
+
−
RK3399, get CPU's working temperature and clock:<br />
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
while true; do
+
cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
−
t1=`cat /sys/class/thermal/thermal_zone0/temp`
+
python3.9 test.py
−
t2=`cat /sys/class/thermal/thermal_zone1/temp`
+
−
f1=`cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq`
+
−
f2=`cat /sys/devices/system/cpu/cpu4/cpufreq/cpuinfo_cur_freq`
+
−
echo "$t1 $t2  $f1 $f2"
+
−
sleep 1
+
−
#cat /sys/class/thermal/thermal_zone[01]/temp
+
−
#cat /sys/devices/system/cpu/cpu[04]/cpufreq/cpuinfo_cur_freq
+
−
done
+
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
output:
 +
<syntaxhighlight lang="text">
 +
W rknn-toolkit-lite2 version: 2.3.0
 +
--> Load RKNN model
 +
done
 +
--> Init runtime environment
 +
I RKNN: [07:08:29.756] RKNN Runtime Information, librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
 +
I RKNN: [07:08:29.757] RKNN Driver Information, version: 0.9.8
 +
I RKNN: [07:08:29.762] RKNN Model Information, version: 6, toolkit version: 2.3.0(compiler version: 2.3.0 (c949ad889d@2024-11-07T11:39:30)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
 +
W RKNN: [07:08:29.814] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
 +
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
 +
done
 +
--> Running model
 +
resnet18
 +
-----TOP 5-----
 +
[812] score:0.999680 class:"space shuttle"
 +
[404] score:0.000249 class:"airliner"
 +
[657] score:0.000013 class:"missile"
 +
[466] score:0.000009 class:"bullet train, bullet"
 +
[895] score:0.000008 class:"warplane, military plane"
  
−
==H3==
+
done
−
===CPU DVFS===
+
</syntaxhighlight>
−
====Linux-4.14====
+
==RKLLM and Large Language Models==
−
All H3 based boards have the same DVFS setting and the setting is in the "arch/arm/boot/dts/sun8i-h3-nanopi.dtsi" file:
+
===Model Conversion (PC Side)===
 +
The Hugging Face-format large language model needs to be converted into an RKLLM model to enable loading and usage on the Rockchip NPU platform. The conversion process must be performed on a PC, with Ubuntu 20.04 as the recommended operating system. A Docker container can also be used.
 +
====Installing RKLLM-Toolkit====
 +
<syntaxhighlight lang="text">
 +
sudo apt-get update
 +
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
 +
cd ~
 +
git clone https://github.com/airockchip/rknn-llm
 +
pip3 install rknn-llm/rkllm-toolkit/packages/rkllm_toolkit-1.1.4-cp38-cp38-linux_x86_64.whl
 +
</syntaxhighlight>
 +
====Download and Convert the Model====
 +
Use the git clone command to download the required model from the Hugging Face repository. Update test.py to point to the downloaded model, then execute test.py to start the model conversion process:
 +
<syntaxhighlight lang="text">
 +
cd ~/rknn-llm/rkllm-toolkit/examples/
 +
git lfs install
 +
git clone https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
 +
sed -i 's|^modelpath\s*=.*|modelpath = "./DeepSeek-R1-Distill-Qwen-1.5B"|' test.py
 +
sed -i 's|^dataset\s*=.*|dataset = None|' test.py
 +
python3 test.py
 +
</syntaxhighlight>
 +
The output information of the model conversion is as follows:
 +
<syntaxhighlight lang="text">
 +
ubuntu@20de568f4011:~/rknn-llm/rkllm-toolkit/examples$ python3 test.py
 +
INFO: rkllm-toolkit version: 1.1.4
 +
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
 +
Optimizing model: 100%|██████████████████████████████████████████████████████████████████████████████████████| 28/28 [01:33<00:00,  3.35s/it]
 +
Building model: 100%|█████████████████████████████████████████████████████████████████████████████████████| 399/399 [00:03<00:00, 102.43it/s]
 +
WARNING: The bos token has two ids: 151646 and 151643, please ensure that the bos token ids in config.json and tokenizer_config.json are consistent!
 +
INFO: The token_id of bos is set to 151646
 +
INFO: The token_id of eos is set to 151643
 +
INFO: The token_id of pad is set to 151643
 +
Converting model: 100%|███████████████████████████████████████████████████████████████████████████████| 339/339 [00:00<00:00, 5642337.52it/s]
 +
INFO: Exporting the model, please wait ....
 +
[=================================================>] 597/597 (100%)
 +
INFO: Model has been saved to ./qwen.rkllm!
 +
</syntaxhighlight>
 +
If successful, the file qwen.rkllm will be generated. Rename it to DeepSeek-R1-Distill-Qwen-1.5B.rkllm and upload it to the development board:
 +
<syntaxhighlight lang="text">
 +
mv qwen.rkllm DeepSeek-R1-Distill-Qwen-1.5B.rkllm
 +
scp DeepSeek-R1-Distill-Qwen-1.5B.rkllm root@NanoPC-T6:/home/pi
 +
</syntaxhighlight>
 +
===Compile and run demo application (On Development Board)===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
&cpu0 {
+
sudo apt-get update
−
operating-points = <
+
sudo apt-get install -y gcc g++ make cmake
−
1008000 1300000
+
−
816000 1100000
+
−
624000 1100000
+
−
480000 1100000
+
−
312000 1100000
+
−
240000 1100000
+
−
120000 1100000
+
−
>;
+
−
#cooling-cells = <2>;
+
−
cooling-min-level = <0>;
+
−
cooling-max-level = <6>;
+
−
cpu0-supply = <&vdd_cpux>;
+
−
};
+
  
−
&cpu_thermal {
+
# get rknn-llm
−
trips {
+
cd ~
−
cpu_warm: cpu_warm {
+
[ -d rknn-llm ] || git clone https://github.com/airockchip/rknn-llm
−
temperature = <65000>;
+
cd rknn-llm/examples/rkllm_api_demo
−
hysteresis = <2000>;
+
chmod +x ./build-linux.sh
−
type = "passive";
+
sed -i 's/^GCC_COMPILER_PATH=.*/GCC_COMPILER_PATH=aarch64-linux-gnu/' ./build-linux.sh
−
};
+
./build-linux.sh
−
cpu_hot: cpu_hot {
+
−
temperature = <75000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_very_hot: cpu_very_hot {
+
−
temperature = <90000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_crit: cpu_crit {
+
−
temperature = <105000>;
+
−
hysteresis = <2000>;
+
−
type = "critical";
+
−
};
+
−
};
+
  
−
cooling-maps {
+
# run llm_demo
−
cpu_warm_limit_cpu {
+
cd ./build/build_linux_aarch64_Release/
−
trip = <&cpu_warm>;
+
export LD_LIBRARY_PATH=~/rknn-llm/rkllm-runtime/Linux/librkllm_api/aarch64:$LD_LIBRARY_PATH
−
cooling-device = <&cpu0 THERMAL_NO_LIMIT 1>;
+
ulimit -HSn 10240
−
};
+
# max_new_tokens = 512, max_context_len = 1024
−
cpu_hot_limit_cpu {
+
taskset f0 ./llm_demo ~/DeepSeek-R1-Distill-Qwen-1.5B.rkllm 512 1024
−
trip = <&cpu_hot>;
+
−
cooling-device = <&cpu0 2 3>;
+
−
};
+
−
cpu_very_hot_limit_cpu {
+
−
trip = <&cpu_very_hot>;
+
−
cooling-device = <&cpu0 5 THERMAL_NO_LIMIT>;
+
−
};
+
−
};
+
−
};
+
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
Output:
 +
<syntaxhighlight lang="text">
 +
rkllm init start
 +
I rkllm: rkllm-runtime version: 1.1.4, rknpu driver version: 0.9.8, platform: RK3588
  
−
===CPU ID===
+
rkllm init success
−
====Linux-4.14====
+
−
{{FriendlyCoreAllwinner-CHIPID}}
+
  
−
===CPU thermal===
+
**********************可输入以下问题对应序号获取回答/或自定义输入********************
−
====Linux-4.14====
+
−
<syntaxhighlight lang="text">
+
−
$ cat /sys/class/thermal/thermal_zone0/temp
+
−
30489
+
−
</syntaxhighlight>
+
−
The reading means the working temperature is 30.489 degrees Celsius.
+
  
−
==H5==
+
[0] 把下面的现代文翻译成文言文: 到了春风和煦，阳光明媚的时候，湖面平静，没有惊涛骇浪，天色湖光相连，一片碧绿，广阔无际；沙洲上的鸥鸟，时而飞翔，时而停歇，美丽的鱼游来游去，岸上与小洲上的花草，青翠欲滴。
−
===CPU DVFS===
+
[1] 以咏梅为题目，帮我写一首古诗，要求包含梅花、白雪等元素。
−
====Linux-4.14====
+
[2] 上联: 江边惯看千帆过
−
For H5 based boards different boards have different DVFS settings.
+
[3] 把这句话翻译成中文: Knowledge can be acquired from many sources. These include books, teachers and practical experience, and each has its own advantages. The knowledge we gain from books and formal education enables us to learn about things that we have no opportunity to experience in daily life. We can also develop our analytical skills and learn how to view and interpret the world around us in different ways. Furthermore, we can learn from the past by reading books. In this way, we won't repeat the mistakes of others and can build on their achievements.
−
Let's take the NanoPi NEO2 as an example the setting is in the "arch/arm64/boot/dts/allwinner/sun50i-h5-nanopi-neo2.dts" file:
+
[4] 把这句话翻译成英文: RK3588是新一代高端处理器，具有高算力、低功耗、超强多媒体、丰富数据接口等特点
−
<syntaxhighlight lang="text">
+
−
&cpu0 {
+
−
operating-points = <
+
−
1008000 1100000
+
−
816000 1100000
+
−
624000 1100000
+
−
480000 1100000
+
−
312000 1100000
+
−
240000 1100000
+
−
120000 1100000
+
−
>;
+
−
#cooling-cells = <2>;
+
−
cooling-min-level = <0>;
+
−
cooling-max-level = <6>;
+
−
cpu0-supply = <&vdd_cpux>;
+
−
};
+
  
−
&cpu_thermal {
+
*************************************************************************
−
trips {
+
−
cpu_warm: cpu_warm {
+
−
temperature = <65000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_hot: cpu_hot {
+
−
temperature = <75000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_very_hot: cpu_very_hot {
+
−
temperature = <90000>;
+
−
hysteresis = <2000>;
+
−
type = "passive";
+
−
};
+
−
cpu_crit: cpu_crit {
+
−
temperature = <105000>;
+
−
hysteresis = <2000>;
+
−
type = "critical";
+
−
};
+
−
};
+
  
−
cooling-maps {
 
−
cpu_warm_limit_cpu {
 
−
trip = <&cpu_warm>;
 
−
cooling-device = <&cpu0 THERMAL_NO_LIMIT 1>;
 
−
};
 
−
cpu_hot_limit_cpu {
 
−
trip = <&cpu_hot>;
 
−
cooling-device = <&cpu0 2 3>;
 
−
};
 
−
cpu_very_hot_limit_cpu {
 
−
trip = <&cpu_very_hot>;
 
−
cooling-device = <&cpu0 5 THERMAL_NO_LIMIT>;
 
−
};
 
−
};
 
−
};
 
−
</syntaxhighlight>
 
  
−
===CPU ID===
+
user: 天空为什么是蓝色的?
−
====Linux-4.14====
+
robot: 这个问题看起来很有趣，但我不太确定。我想知道天空的颜色是什么原因导致的，以及是否有其他因素影响了它的颜色。
−
{{FriendlyCoreAllwinner-CHIPID}}
+
首先，我想到的是太阳本身的颜色。太阳看起来像一个红色的东西，对吧？那是不是和天空的颜色有关呢？
 +
然后，我还记得以前学过一点关于大气层的知识，说大气层是透明的，但有一定的折射作用。这可能影响了我们看到的天空颜色。
 +
另外，我还听说过光的反射和折射的一些定律，比如费马原理，可能也与天空的颜色有关。
 +
不过，我还不太确定这些因素是如何相互作用的，以及它们最终导致了天空为什么会是蓝色的。
  
−
===CPU thermal===
+
也许我可以先从太阳本身的颜色开始分析。太阳看起来红色，是因为它发出的光大部分在红光区，而蓝光和绿光被大气层吸收或散射，所以我们看到的是红色。那如果太阳本身是蓝色的话，会不会影响天空的颜色呢？
−
====Linux-4.14====
+
另外，我还记得有人说过，天空的颜色可能与地球上的天气状况有关，比如阳光是否直射、云的类型等等。
 +
不过，这些信息我还不太确定，需要进一步查证。
 +
 
 +
总的来说，我觉得天空的颜色是由多种因素共同作用的结果，包括太阳的颜色、大气层的折射和反射、光的反射定律以及天气状况等。但具体来说，为什么天空会是蓝色呢？这可能涉及到光的折射和反射现象，以及地球表面的光线分布。
 +
</think>
 +
 
 +
天空呈现蓝色的原因主要归因于光的折射和反射现象，以及地球表面光线分布的影响。以下是详细的解释：
 +
 
 +
1. **太阳的颜色**：太阳本身看起来红色，这是因为太阳发出的大部分光在红光区，而蓝光和绿光被大气层吸收或散射，所以我们看到的是红色。
 +
 
 +
2. **大气层的作用**：
 +
  - 大气层是透明的，但具有折射作用。当阳光穿过大气层时，部分光线会被折射，导致天空呈现蓝色。
 +
  - 这种现象称为“大气折射”或“海市蜃楼效应”。不同颜色的光在不同介质中的折射率不同，导致它们在空气中传播时发生偏折。
 +
 
 +
3. **光的反射和折射**：
 +
  - 光线在空气中的传播遵循费马原理，即光线会尽可能地快速到达目标。这包括反射和折射。
 +
  - 太阳光穿过大气层后，在地球表面反射，而部分光被折射进入天空，
 +
 
 +
user:
 +
</syntaxhighlight>
 +
===Useful Resources===
 +
* https://github.com/airockchip/rknn-llm
 +
* https://github.com/wudingjian/rkllm_chat
 +
==Doc==
 +
https://github.com/rockchip-linux/rknpu2/tree/master/doc
 +
==Other==
 +
===View NPU Load===
 +
<syntaxhighlight lang="text">
 +
sudo cat /sys/kernel/debug/rknpu/load
 +
</syntaxhighlight>
 +
===Set NPU freq===
 +
<syntaxhighlight lang="text">
 +
echo userspace | sudo tee /sys/class/devfreq/fdab0000.npu/governor > /dev/null
 +
echo 800000000 | sudo tee /sys/class/devfreq/fdab0000.npu/min_freq > /dev/null
 +
echo 1000000000 | sudo tee /sys/class/devfreq/fdab0000.npu/max_freq > /dev/null
 +
</syntaxhighlight>
 +
===View NPU freq===
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
−
$ cat /sys/class/thermal/thermal_zone0/temp
+
cat /sys/class/devfreq/fdab0000.npu/cur_freq
−
30489
+
 
</syntaxhighlight>
 
</syntaxhighlight>
−
The reading means the working temperature is 30.489 degrees Celsius.
 

Latest revision as of 06:02, 7 February 2025

查看中文

1 Earlier version RKNPU2 SDK

Link to → v1.5.2

2 Check NPU driver version

$ sudo cat /sys/kernel/debug/rknpu/version
RKNPU driver: v0.9.8

3 RKNN Quick Example

3.1 OS

Tested on the following OS:

3.1.1 Debian11 (bullseye)

  • rk3588-sd-debian-bullseye-desktop-6.1-arm64-20250117.img.gz

3.1.2 Ubuntu20 (focal)

  • rk3588-sd-ubuntu-focal-desktop-6.1-arm64-20250117.img.gz

3.2 Installing RKNN Runtime

cd ~
export GIT_SSL_NO_VERIFY=1
git clone https://github.com/airockchip/rknn-toolkit2.git
cd rknn-toolkit2/rknpu2
sudo cp ./runtime/Linux/librknn_api/aarch64/* /usr/lib
sudo cp ./runtime/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/
sudo cp ./runtime/Linux/librknn_api/include/* /usr/include/

3.3 Check rknn version

$ strings /usr/bin/rknn_server |grep 'build@'
2.3.0 (e80ac5c build@2024-11-07T12:52:53)
rknn_server version: 2.3.0 (e80ac5c build@2024-11-07T12:52:53)
$ strings /usr/lib/librknnrt.so |grep 'librknnrt version:'
librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)

3.4 C++ Example: Build and run the YOLOv5 demo

sudo apt-get update
sudo apt-get install -y gcc g++ make cmake
 
# fix broken link
cd ~/rknn-toolkit2/rknpu2/examples/3rdparty/mpp/Linux/aarch64
rm -f librockchip_mpp.so librockchip_mpp.so.1
ln -s librockchip_mpp.so.0 librockchip_mpp.so
ln -s librockchip_mpp.so.0 librockchip_mpp.so.1
 
cd ~/rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo
chmod +x ./build-linux.sh
sudo ln -s /usr/bin/gcc /usr/bin/aarch64-gcc
sudo ln -s /usr/bin/g++ /usr/bin/aarch64-g++
export GCC_COMPILER=aarch64
./build-linux.sh -t rk3588 -a aarch64 -b Release
cd install/rknn_yolov5_demo_Linux
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg

Transfer the generated out.jpg to PC to view the result: 

scp out.jpg xxx@YourIP:/tmp/

Rknn yolov5 demo out.jpg

3.5 Python Example: Install and Test RKNN Toolkit Lite2 on Debian11

3.5.1 Install RKNN Toolkit Lite2

sudo apt-get update
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
cd ~/rknn-toolkit2
pip3 install ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.3.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple/

3.5.2 Run the python example

$ cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
$ python3 test.py
W rknn-toolkit-lite2 version: 2.3.0
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [06:29:43.602] RKNN Runtime Information, librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
I RKNN: [06:29:43.602] RKNN Driver Information, version: 0.9.8
I RKNN: [06:29:43.602] RKNN Model Information, version: 6, toolkit version: 2.3.0(compiler version: 2.3.0 (c949ad889d@2024-11-07T11:39:30)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
W RKNN: [06:29:43.617] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
done
--> Running model
resnet18
-----TOP 5-----
[812] score:0.999680 class:"space shuttle"
[404] score:0.000249 class:"airliner"
[657] score:0.000013 class:"missile"
[466] score:0.000009 class:"bullet train, bullet"
[895] score:0.000008 class:"warplane, military plane"
 
done

3.6 Python Example: Install and Test RKNN Toolkit Lite2 on Ubuntu 20.04 (Focal)

3.6.1 Compile and install python3.9

sudo apt install build-essential libssl-dev libffi-dev software-properties-common \
    libbz2-dev libncurses-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev \
    tk-dev libgdbm-compat-dev libreadline-dev
 
wget https://www.python.org/ftp/python/3.9.21/Python-3.9.21.tar.xz
tar -xvf Python-3.9.21.tar.xz
cd Python-3.9.21/
./configure --enable-optimizations
make -j$(nproc)
sudo make install

3.6.2 Install RKNN Toolkit Lite2

python3.9 -m pip install --upgrade pip
python3.9 -m pip install opencv-python
 
cd ~
[ -d rknn-toolkit2 ] || git clone https://github.com/airockchip/rknn-toolkit2.git \
    --depth 1 -b master
cd rknn-toolkit2
python3.9 -m pip install \
    ./rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.3.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl \
    -i https://pypi.tuna.tsinghua.edu.cn/simple/

3.6.3 Run the python example

cd ~/rknn-toolkit2/rknn-toolkit-lite2/examples/resnet18/
python3.9 test.py

output: 

W rknn-toolkit-lite2 version: 2.3.0
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [07:08:29.756] RKNN Runtime Information, librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
I RKNN: [07:08:29.757] RKNN Driver Information, version: 0.9.8
I RKNN: [07:08:29.762] RKNN Model Information, version: 6, toolkit version: 2.3.0(compiler version: 2.3.0 (c949ad889d@2024-11-07T11:39:30)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW, model inference type: static_shape
W RKNN: [07:08:29.814] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
done
--> Running model
resnet18
-----TOP 5-----
[812] score:0.999680 class:"space shuttle"
[404] score:0.000249 class:"airliner"
[657] score:0.000013 class:"missile"
[466] score:0.000009 class:"bullet train, bullet"
[895] score:0.000008 class:"warplane, military plane"
 
done

4 RKLLM and Large Language Models

4.1 Model Conversion (PC Side)

The Hugging Face-format large language model needs to be converted into an RKLLM model to enable loading and usage on the Rockchip NPU platform. The conversion process must be performed on a PC, with Ubuntu 20.04 as the recommended operating system. A Docker container can also be used.

4.1.1 Installing RKLLM-Toolkit

sudo apt-get update
sudo apt-get install -y python3-dev python3-numpy python3-opencv python3-pip
cd ~
git clone https://github.com/airockchip/rknn-llm
pip3 install rknn-llm/rkllm-toolkit/packages/rkllm_toolkit-1.1.4-cp38-cp38-linux_x86_64.whl

4.1.2 Download and Convert the Model

Use the git clone command to download the required model from the Hugging Face repository. Update test.py to point to the downloaded model, then execute test.py to start the model conversion process: 

cd ~/rknn-llm/rkllm-toolkit/examples/
git lfs install
git clone https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
sed -i 's|^modelpath\s*=.*|modelpath = "./DeepSeek-R1-Distill-Qwen-1.5B"|' test.py
sed -i 's|^dataset\s*=.*|dataset = None|' test.py
python3 test.py

The output information of the model conversion is as follows: 

ubuntu@20de568f4011:~/rknn-llm/rkllm-toolkit/examples$ python3 test.py
INFO: rkllm-toolkit version: 1.1.4
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
Optimizing model: 100%|██████████████████████████████████████████████████████████████████████████████████████| 28/28 [01:33<00:00,  3.35s/it]
Building model: 100%|█████████████████████████████████████████████████████████████████████████████████████| 399/399 [00:03<00:00, 102.43it/s]
WARNING: The bos token has two ids: 151646 and 151643, please ensure that the bos token ids in config.json and tokenizer_config.json are consistent!
INFO: The token_id of bos is set to 151646
INFO: The token_id of eos is set to 151643
INFO: The token_id of pad is set to 151643
Converting model: 100%|███████████████████████████████████████████████████████████████████████████████| 339/339 [00:00<00:00, 5642337.52it/s]
INFO: Exporting the model, please wait ....
[=================================================>] 597/597 (100%)
INFO: Model has been saved to ./qwen.rkllm!

If successful, the file qwen.rkllm will be generated. Rename it to DeepSeek-R1-Distill-Qwen-1.5B.rkllm and upload it to the development board: 

mv qwen.rkllm DeepSeek-R1-Distill-Qwen-1.5B.rkllm
scp DeepSeek-R1-Distill-Qwen-1.5B.rkllm root@NanoPC-T6:/home/pi

4.2 Compile and run demo application (On Development Board)

sudo apt-get update
sudo apt-get install -y gcc g++ make cmake
 
# get rknn-llm
cd ~
[ -d rknn-llm ] || git clone https://github.com/airockchip/rknn-llm
cd rknn-llm/examples/rkllm_api_demo
chmod +x ./build-linux.sh
sed -i 's/^GCC_COMPILER_PATH=.*/GCC_COMPILER_PATH=aarch64-linux-gnu/' ./build-linux.sh
./build-linux.sh
 
# run llm_demo
cd ./build/build_linux_aarch64_Release/
export LD_LIBRARY_PATH=~/rknn-llm/rkllm-runtime/Linux/librkllm_api/aarch64:$LD_LIBRARY_PATH
ulimit -HSn 10240
# max_new_tokens = 512, max_context_len = 1024
taskset f0 ./llm_demo ~/DeepSeek-R1-Distill-Qwen-1.5B.rkllm 512 1024

Output: 

rkllm init start
I rkllm: rkllm-runtime version: 1.1.4, rknpu driver version: 0.9.8, platform: RK3588
 
rkllm init success
 
**********************可输入以下问题对应序号获取回答/或自定义输入********************
 
[0] 把下面的现代文翻译成文言文: 到了春风和煦，阳光明媚的时候，湖面平静，没有惊涛骇浪，天色湖光相连，一片碧绿，广阔无际；沙洲上的鸥鸟，时而飞翔，时而停歇，美丽的鱼游来游去，岸上与小洲上的花草，青翠欲滴。
[1] 以咏梅为题目，帮我写一首古诗，要求包含梅花、白雪等元素。
[2] 上联: 江边惯看千帆过
[3] 把这句话翻译成中文: Knowledge can be acquired from many sources. These include books, teachers and practical experience, and each has its own advantages. The knowledge we gain from books and formal education enables us to learn about things that we have no opportunity to experience in daily life. We can also develop our analytical skills and learn how to view and interpret the world around us in different ways. Furthermore, we can learn from the past by reading books. In this way, we won't repeat the mistakes of others and can build on their achievements.
[4] 把这句话翻译成英文: RK3588是新一代高端处理器，具有高算力、低功耗、超强多媒体、丰富数据接口等特点
 
*************************************************************************
 
 
user: 天空为什么是蓝色的?
robot: 这个问题看起来很有趣，但我不太确定。我想知道天空的颜色是什么原因导致的，以及是否有其他因素影响了它的颜色。
首先，我想到的是太阳本身的颜色。太阳看起来像一个红色的东西，对吧？那是不是和天空的颜色有关呢？
然后，我还记得以前学过一点关于大气层的知识，说大气层是透明的，但有一定的折射作用。这可能影响了我们看到的天空颜色。
另外，我还听说过光的反射和折射的一些定律，比如费马原理，可能也与天空的颜色有关。
不过，我还不太确定这些因素是如何相互作用的，以及它们最终导致了天空为什么会是蓝色的。
 
也许我可以先从太阳本身的颜色开始分析。太阳看起来红色，是因为它发出的光大部分在红光区，而蓝光和绿光被大气层吸收或散射，所以我们看到的是红色。那如果太阳本身是蓝色的话，会不会影响天空的颜色呢？
另外，我还记得有人说过，天空的颜色可能与地球上的天气状况有关，比如阳光是否直射、云的类型等等。
不过，这些信息我还不太确定，需要进一步查证。
 
总的来说，我觉得天空的颜色是由多种因素共同作用的结果，包括太阳的颜色、大气层的折射和反射、光的反射定律以及天气状况等。但具体来说，为什么天空会是蓝色呢？这可能涉及到光的折射和反射现象，以及地球表面的光线分布。
</think>
 
天空呈现蓝色的原因主要归因于光的折射和反射现象，以及地球表面光线分布的影响。以下是详细的解释：
 
1. **太阳的颜色**：太阳本身看起来红色，这是因为太阳发出的大部分光在红光区，而蓝光和绿光被大气层吸收或散射，所以我们看到的是红色。
 
2. **大气层的作用**：
   - 大气层是透明的，但具有折射作用。当阳光穿过大气层时，部分光线会被折射，导致天空呈现蓝色。
   - 这种现象称为“大气折射”或“海市蜃楼效应”。不同颜色的光在不同介质中的折射率不同，导致它们在空气中传播时发生偏折。
 
3. **光的反射和折射**：
   - 光线在空气中的传播遵循费马原理，即光线会尽可能地快速到达目标。这包括反射和折射。
   - 太阳光穿过大气层后，在地球表面反射，而部分光被折射进入天空，
 
user:

4.3 Useful Resources

5 Doc

https://github.com/rockchip-linux/rknpu2/tree/master/doc

6 Other

6.1 View NPU Load

sudo cat /sys/kernel/debug/rknpu/load

6.2 Set NPU freq

echo userspace | sudo tee /sys/class/devfreq/fdab0000.npu/governor > /dev/null
echo 800000000 | sudo tee /sys/class/devfreq/fdab0000.npu/min_freq > /dev/null
echo 1000000000 | sudo tee /sys/class/devfreq/fdab0000.npu/max_freq > /dev/null

6.3 View NPU freq

cat /sys/class/devfreq/fdab0000.npu/cur_freq
Retrieved from "https://wiki.friendlyelec.com/wiki/index.php?title=NPU&oldid=26726"