Difference between revisions of "How to use overlayfs on Linux"

From FriendlyELEC WiKi
Jump to: navigation, search
(updated by API)
 
(updated by API)
 
(19 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[How to build FriendlyELEC FriendlyWRT|English]]
+
[[How to use overlayfs on Linux/zh|查看中文]]
  
==FriendlyWRT简介==
+
==What Is OverlayFS==
FriendlyWRT是友善电子基于OpenWrt定制的系统,完全开源,用于企业物联网二次开发,个人定制NAS,家庭物联网网关等。
+
OverlayFS is a union mount filesystem implementation for Linux. It allows a virtual merge of two partitions, while keeping their actual contents separate. One partition is the rootfs partition and the other is the data partition. It has the following advantages:<br />
TODO <br />
+
1) you can easily restore a system's factory settings by formatting the data partition;<br />
==下载体验版固件==
+
2) you can still boot your system since the rootfs is read-only even when the data partition cannot be correctly mounted due to unexpected shutdown.<br />
TODO <br />
+
解压后用 dd 或者 win32image 烧写到 SD 卡。
+
==帐户与密码==
+
用户名: root<br />
+
密码: fa<br />
+
<br />
+
==获取项目源代码==
+
===安装 repo 工具===
+
首先需要安装 repo 工具:
+
<syntaxhighlight lang="bash">
+
git clone https://github.com/friendlyarm/repo
+
cp repo/repo /usr/bin/
+
</syntaxhighlight>
+
===下载项目源代码===
+
有以下两种途径获取项目源代码,中国大陆用户建议使用方法1的途径下载:
+
====方法一:使用网盘里的repo压缩包====
+
网盘下载地址: [http://download.friendlyarm.com/{{#replace:{{#replace:{{BASEPAGENAME}}| |}}|/zh|}} 点击进入]<br />
+
文件位于网盘的以下路径:sources/friendlywrt-YYYYMMDD.tar (YYYYMMDD表示打包的日期)<br />
+
从网盘中获取的 repo 压缩包在解压之后,需要执行一下以下命令做一次解包动作:
+
<syntaxhighlight lang="bash">
+
tar xvf /path/to/netdisk/sources/friendlywrt-YYYYMMDD.tar
+
cd friendlywrt
+
repo sync -l
+
</syntaxhighlight>
+
用此方法得到的源代码版本是repo打包时的版本,如果想拉取到官方最新的版本,可以去掉 -l 参数,执行一次 repo sync, 例如:
+
<syntaxhighlight lang="bash">
+
cd friendlywrt
+
repo sync
+
</syntaxhighlight>
+
====方法二:从github下载====
+
<syntaxhighlight lang="bash">
+
mkdir friendlywrt
+
cd friendlywrt
+
repo init -u https://github.com/friendlyarm/friendlywrt_manifests -b master -m h3.xml --repo-url=https://github.com/friendlyarm/repo
+
repo sync -c
+
</syntaxhighlight>
+
====同步项目到最新版本====
+
<syntaxhighlight lang="bash">
+
cd friendlywrt
+
repo sync -c
+
</syntaxhighlight>
+
同步过程中,由于网络原因出现中断,可以使用下面脚本同步代码:
+
<syntaxhighlight lang="bash">
+
#! /bin/bash
+
repo sync -c
+
while [ $? -ne 0 ];
+
do 
+
    repo sync -c
+
done
+
</syntaxhighlight>
+
  
==如何编译==
+
==FriendlyELEC's Systems That Support OverlayFS==
===搭建编译环境===
+
===Hardware Systems===
在PC的Ubuntu系统下 (建议使用版本18.04-64bit),执下以下命令安装所需的软件:
+
H3, H5, S5P4418, S5P6818, RK3399, RK3328, RK3568, RK3588 based boards
 +
===OS Systems===
 +
All Linux-based systems
 +
===Scope of Application===
 +
This document is only applicable to products using Rockchip platform. If you are using other platforms, please click on this link: [[How to use overlayfs on S5Pxxxx,H3,H5 platform]]
 +
==How to Check Whether OverlayFS Is Working==
 +
Run the df command. If the "/" partition is mounted as "overlay" it means OverlayFS is working;
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
wget -O - https://raw.githubusercontent.com/friendlyarm/build-env-on-ubuntu-bionic/master/install.sh | bash
+
pi@NanoPi-R6C:/etc$ df -h
 +
Filesystem      Size  Used Avail Use% Mounted on
 +
tmpfs          792M  2.2M  790M  1% /run
 +
overlay          25G  13G  11G  53% /
 +
tmpfs          3.9G    0  3.9G  0% /dev/shm
 +
tmpfs          5.0M  4.0K  5.0M  1% /run/lock
 +
tmpfs          793M  116K  793M  1% /run/user/1000
 
</syntaxhighlight>
 
</syntaxhighlight>
 
+
==Partition layout when using OverlayFS==
===查看帮助===
+
The user data will be composed of the rootfs partition and the userdata partition, corresponding to the image files rootfs.img and userdata.img. <br />
不带参数执行 build.sh,默认是打印帮助信息:
+
Enter the following command to view the partition layout:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
./build.sh
+
View eMMC partition layout: sudo parted -s /dev/mmcblk2 unit MiB print
 +
View SD card partition layout: sudo parted -s /dev/mmcblk0 unit MiB print
 
</syntaxhighlight>
 
</syntaxhighlight>
显示的帮助信息如下:
+
The output is shown as follows:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
USAGE: ./build.sh <parameter>
+
pi@NanoPi-R6C:/etc$ sudo apt install parted
 +
pi@NanoPi-R6C:/etc$ sudo parted /dev/mmcblk0 print
 +
Model: SD SR32G (sd/mmc)
 +
Disk /dev/mmcblk0: 31.9GB
 +
Sector size (logical/physical): 512B/512B
 +
Partition Table: gpt
 +
Disk Flags:
  
# 选择目标的硬件平台:
+
Number  Start   End    Size    File system  Name      Flags
   ./build.sh friendlyelec_h3_series.mk
+
1      8389kB  12.6MB  4194kB              uboot
 
+
2      12.6MB  16.8MB  4194kB              misc
# 编译各个模块:
+
3      16.8MB  21.0MB  4194kB              dtbo
  ./build.sh all                -编译所有组件
+
4      21.0MB  37.7MB  16.8MB              resource
  ./build.sh uboot              -单独编译uboot
+
5      37.7MB  79.7MB  41.9MB              kernel
  ./build.sh kernel             -单独编译kernel
+
6      79.7MB  113MB   33.6MB              boot
   ./build.sh friendlywrt        -单独编译friendlywrt
+
7      113MB  147MB   33.6MB              recovery
   ./build.sh sd-img            -生成sd启动的镜像文件
+
8      147MB   4173MB  4027MB  ext4        rootfs
   ./build.sh emmc-img          -生成用于安装到emmc的镜像文件 (镜像文件需要dd到sd卡,通过sd卡启动安装程序)
+
9      4173MB  31.9GB  27.7GB  ext4        userdata
# 清理:
+
  ./build.sh cleanall
+
 
</syntaxhighlight>
 
</syntaxhighlight>
===全自动方式编译===
+
the rootfs partition is mounted as read-only, while userdata is mounted as read-write. The rootfs partition stores fixed system data, and any subsequent writes to the root directory will be written to the userdata partition. Therefore, formatting userdata is equivalent to restoring factory settings.<br />
初次编译,需要选择目标的硬件平台,当前只有H3可用,后续会加入更多的硬件支持:
+
== Common Operations ==
 +
'''Notes:'''<br/>
 +
* These operations will erase user data, so be sure to backup your data in advance.<br>
 +
* The OS image needs to be updated to version 2023/03/14 or later, or only the boot.img needs to be updated.<br>
 +
* The device nodes /dev/mmcblkX that appear in the commands are fictitious nodes, and need to be changed to the real devices. The device node for eMMC is /dev/mmcblk2, and the device node for TF card is /dev/mmcblk0.<br>
 +
===View the current partition layout===
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
./build.sh friendlyelec_h3_series.mk
+
sudo apt update
 +
sudo apt install parted
 +
export DEV=/dev/mmcblkX # Need to change to real device
 +
sudo parted -s ${DEV} unit MiB print
 
</syntaxhighlight>
 
</syntaxhighlight>
执行上述命令,会自动编译所有组件,其中包括u-boot, kernel 和 friendlywrt,并生成sd启动的烧写镜像文件。
+
===Disable OverlayFS feature===
===编译单独的模块===
+
Create a file named ".init_wipedata" in the root directory with the content "overlayfs=disable" and then reboot:
====kernel====
+
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
./build.sh kernel
+
sudo passwd root  # Create a password for the root user, if not done before
 +
su - root -c 'echo "overlayfs=disable" > /.init_wipedata'
 +
sudo reboot
 
</syntaxhighlight>
 
</syntaxhighlight>
====u-boot====
+
=== Resize the userdata partition and create an additional partition===
 +
* View current partition layout
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
./build.sh uboot
+
sudo apt update
 +
sudo apt install parted fdisk
 +
export DEV=/dev/mmcblkX # Need to change to real device
 +
sudo parted -s ${DEV} unit MiB print
 
</syntaxhighlight>
 
</syntaxhighlight>
====friendlywrt====
+
* Resize the userdata partition
 +
Here is an example of resizing to 8G, create a file named ".init_wipedata" in the root directory, with the content "overlayfs=enable userdata=8096", and then Reboot, where userdata= is followed by the new userdata partition size in MB, the command is as follows:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
./build.sh friendlywrt
+
su - root -c 'echo "overlayfs=enable userdata=8096" > /.init_wipedata'
 +
sudo reboot
 
</syntaxhighlight>
 
</syntaxhighlight>
===生成sdcard固件===
+
After reboot, you can see that the userdata partition has been resized to 8GB:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
sudo ./build.sh sd-img
+
export DEV=/dev/mmcblkX  # Need to change to real device
 +
sudo parted -s ${DEV} unit MiB print
 
</syntaxhighlight>
 
</syntaxhighlight>
打包img成功后,终端会显示如下信息,可以参考下面的dd命令将img写入sd卡,注意/dev/sdX需要替换成真实的sd卡设备:<br />
+
* Create new partition and format it
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
Run the following for sdcard install:
+
(echo n; echo ""; echo ""; echo ""; echo w) | sudo fdisk ${DEV}
    sudo dd if=out/h3-sd-friendlywrt-4.14-armhf-20190807.img bs=1M of=/dev/sdX
+
NUM=$(sudo parted ${DEV} print | awk 'NF > 1 {p = $1} END {print p}')  # Get the last partition index
 +
sudo mkfs.ext4 ${DEV}p${NUM}
 
</syntaxhighlight>
 
</syntaxhighlight>
===生成emmc (eflasher) 固件===
+
* Mount the new partition to the specified directory
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
sudo ./build.sh emmc-img
+
sudo mkdir -p /oem
 +
sudo blkid ${DEV}p${NUM}
 +
# Note down the UUID
 +
# Configure automatic mounting
 +
sudo vi /etc/fstab
 +
# Add the following to the end of the file (where the UUID needs to be replaced with the real one)
 +
UUID=bbb06fe1-df52-4c7c-b2eb-926b14605fe4 /oem ext4 suid,dev,exec,auto,nouser,async,noatime,nofail 0 0
 +
# Enter the following command to mount the partition
 +
sudo mount /oem
 
</syntaxhighlight>
 
</syntaxhighlight>
打包img成功后,终端会显示如下信息,可以参考下面的dd命令将img写入sd卡,注意/dev/sdX需要替换成真实的sd卡设备:<br />
+
===Restore factory settings===
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
Run the following for sdcard install:
+
su - root -c 'echo "overlayfs=enable" > /.init_wipedata'
    sudo dd if=out/h3-eflasher-friendlywrt-4.14-armhf-20190807.img bs=1M of=/dev/sdX
+
sudo reboot
 
</syntaxhighlight>
 
</syntaxhighlight>
用此sd卡启动eflasher系统,执行eflasher命令,将系统写入emmc,之后就可以拨掉sd卡,从emmc启动friendlywrt了。
+
The next time boot into the ramdisk stage, the userdata partition is formatted and the /var/.init_wipedata file is cleared.
==friendlywrt定制与开发==
+
===Debug===
===源代码目录结构===
+
If you find that the partition has not changed, there may be an error, check if an error message is printed with the following command:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
├── friendlywrt
+
dmesg | grep initfs
│  ├── friendlywrt -> friendlywrt源代码
+
|   └── configs -> 保存预设好的friendlywrt defconfig文件
+
│  ├── build.sh -> 全自动编译脚本
+
│  ├── device/friendlyelec -> 用于适配友善电子开发板的相关文件
+
│  ├── kernel -> 内核
+
│  └── u-boot -> u-boot
+
│  └── scripts -> 打包img的相关脚本
+
 
</syntaxhighlight>
 
</syntaxhighlight>
===配置friendlywrt的软件包===
 
* 使用make menuconfig更改配置
 
<syntaxhighlight lang="bash">
 
cd friendlywrt
 
make menuconfig
 
</syntaxhighlight>
 
* 另存你个人的软件包配置
 
<syntaxhighlight lang="bash">
 
make menuconfig
 
./scripts/diffconfig.sh > ../configs/my_config
 
</syntaxhighlight>
 
* 让以后的编译使用你的friendlywrt配置
 
编辑以下文件:
 
<syntaxhighlight lang="bash">
 
device/friendlyelec/h3/friendlyelec_h3_series.mk
 
</syntaxhighlight>
 
将TARGET_FRIENDLYWRT_CONFIG的值改为你的配置。
 
* 重新编译friendlywrt并制成sd启动镜像
 
<syntaxhighlight lang="bash">
 
./build.sh friendlywrt
 
./build.sh sd-img
 
</syntaxhighlight>
 
===固化个人文件到friendlywrt===
 
将文件或目录放入以下目录即可,打包img时会把它们都打包进去:
 
<syntaxhighlight lang="bash">
 
device/friendlyelec/h3/common-files
 
</syntaxhighlight>
 
也可以放在新建的目录里,然后编辑以下文件:
 
<syntaxhighlight lang="bash">
 
device/friendlyelec/h3/friendlyelec_h3_series.mk
 
</syntaxhighlight>
 
在文件中新增一行 (XXX替换为你的目录名):
 
<syntaxhighlight lang="bash">
 
FRIENDLYWRT_FILES+=(device/friendlyelec/h3/XXX)
 
</syntaxhighlight>
 
===修改friendlywrt系统的默认配置(如网络、防火墙等)===
 
进入如下目录:
 
<syntaxhighlight lang="bash">
 
device/friendlyelec/h3/default-settings/files/root/board
 
</syntaxhighlight>
 
编辑你所用的开发板目录所在的配置文件即可,比如下面是NanoPi-R1的配置文件:
 
<syntaxhighlight lang="bash">
 
NanoPi-R1
 
└── etc
 
    └── config
 
        ├── network
 
        └── system
 
</syntaxhighlight>
 
===更改u-boot和kernel配置===
 
同样是编辑如下文件:
 
<syntaxhighlight lang="bash">
 
device/friendlyelec/h3/friendlyelec_h3_series.mk
 
</syntaxhighlight>
 
可通过以下设置,将uboot的配置改为你自已的:
 
<syntaxhighlight lang="bash">
 
TARGET_UBOOT_CONFIG=nanopi_h3_defconfig
 
</syntaxhighlight>
 
可通过以下设置,将kernel的配置改为你自已的:
 
<syntaxhighlight lang="bash">
 
TARGET_KERNEL_CONFIG=sunxi_defconfig
 
</syntaxhighlight>
 
===更换其他源代码===
 
编辑如下 xml 文件,即可替换 kernel 、uboot以及friendlywrt的源代码:
 
<syntaxhighlight lang="bash">
 
.repo/manifests/h3.xml
 
</syntaxhighlight>
 
想了解 xml 文件各个节点的含义和用法,请参考:[https://gerrit.googlesource.com/git-repo/+/refs/heads/master/docs/manifest-format.md repo Manifest Format]<br />
 
编辑完成后,需要用 repo 命令同步一下,比如更换了 kernel 的源代码,用以下命令同步一次:
 
<syntaxhighlight lang="bash">
 
repo sync --force-sync kernel
 
</syntaxhighlight>
 
同步后单独重新内核即可:
 
<syntaxhighlight lang="bash">
 
./build.sh kernel
 
</syntaxhighlight>
 
 
==常见问题及注意事项==
 
* Ubuntu下不能使用 root 用户来编译,需要使用普通用户
 
* 在 vnc 环境下编译可能会编译失败
 
 
==参考资料==
 
[https://source.android.com/setup/develop/repo repo使用指南]<br />
 

Latest revision as of 09:33, 20 March 2023

查看中文

1 What Is OverlayFS

OverlayFS is a union mount filesystem implementation for Linux. It allows a virtual merge of two partitions, while keeping their actual contents separate. One partition is the rootfs partition and the other is the data partition. It has the following advantages:
1) you can easily restore a system's factory settings by formatting the data partition;
2) you can still boot your system since the rootfs is read-only even when the data partition cannot be correctly mounted due to unexpected shutdown.

2 FriendlyELEC's Systems That Support OverlayFS

2.1 Hardware Systems

H3, H5, S5P4418, S5P6818, RK3399, RK3328, RK3568, RK3588 based boards

2.2 OS Systems

All Linux-based systems

2.3 Scope of Application

This document is only applicable to products using Rockchip platform. If you are using other platforms, please click on this link: How to use overlayfs on S5Pxxxx,H3,H5 platform

3 How to Check Whether OverlayFS Is Working

Run the df command. If the "/" partition is mounted as "overlay" it means OverlayFS is working;

pi@NanoPi-R6C:/etc$ df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           792M  2.2M  790M   1% /run
overlay          25G   13G   11G  53% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           793M  116K  793M   1% /run/user/1000

4 Partition layout when using OverlayFS

The user data will be composed of the rootfs partition and the userdata partition, corresponding to the image files rootfs.img and userdata.img.
Enter the following command to view the partition layout:

View eMMC partition layout: sudo parted -s /dev/mmcblk2 unit MiB print
View SD card partition layout: sudo parted -s /dev/mmcblk0 unit MiB print

The output is shown as follows:

pi@NanoPi-R6C:/etc$ sudo apt install parted
pi@NanoPi-R6C:/etc$ sudo parted /dev/mmcblk0 print
Model: SD SR32G (sd/mmc)
Disk /dev/mmcblk0: 31.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
 
Number  Start   End     Size    File system  Name      Flags
 1      8389kB  12.6MB  4194kB               uboot
 2      12.6MB  16.8MB  4194kB               misc
 3      16.8MB  21.0MB  4194kB               dtbo
 4      21.0MB  37.7MB  16.8MB               resource
 5      37.7MB  79.7MB  41.9MB               kernel
 6      79.7MB  113MB   33.6MB               boot
 7      113MB   147MB   33.6MB               recovery
 8      147MB   4173MB  4027MB  ext4         rootfs
 9      4173MB  31.9GB  27.7GB  ext4         userdata

the rootfs partition is mounted as read-only, while userdata is mounted as read-write. The rootfs partition stores fixed system data, and any subsequent writes to the root directory will be written to the userdata partition. Therefore, formatting userdata is equivalent to restoring factory settings.

5 Common Operations

Notes:

  • These operations will erase user data, so be sure to backup your data in advance.
  • The OS image needs to be updated to version 2023/03/14 or later, or only the boot.img needs to be updated.
  • The device nodes /dev/mmcblkX that appear in the commands are fictitious nodes, and need to be changed to the real devices. The device node for eMMC is /dev/mmcblk2, and the device node for TF card is /dev/mmcblk0.

5.1 View the current partition layout

sudo apt update
sudo apt install parted
export DEV=/dev/mmcblkX # Need to change to real device
sudo parted -s ${DEV} unit MiB print

5.2 Disable OverlayFS feature

Create a file named ".init_wipedata" in the root directory with the content "overlayfs=disable" and then reboot:

sudo passwd root  # Create a password for the root user, if not done before
su - root -c 'echo "overlayfs=disable" > /.init_wipedata'
sudo reboot

5.3 Resize the userdata partition and create an additional partition

  • View current partition layout
sudo apt update
sudo apt install parted fdisk
export DEV=/dev/mmcblkX # Need to change to real device
sudo parted -s ${DEV} unit MiB print
  • Resize the userdata partition

Here is an example of resizing to 8G, create a file named ".init_wipedata" in the root directory, with the content "overlayfs=enable userdata=8096", and then Reboot, where userdata= is followed by the new userdata partition size in MB, the command is as follows:

su - root -c 'echo "overlayfs=enable userdata=8096" > /.init_wipedata'
sudo reboot

After reboot, you can see that the userdata partition has been resized to 8GB:

export DEV=/dev/mmcblkX  # Need to change to real device
sudo parted -s ${DEV} unit MiB print
  • Create new partition and format it
(echo n; echo ""; echo ""; echo ""; echo w) | sudo fdisk ${DEV}
NUM=$(sudo parted ${DEV} print | awk 'NF > 1 {p = $1} END {print p}')  # Get the last partition index
sudo mkfs.ext4 ${DEV}p${NUM}
  • Mount the new partition to the specified directory
sudo mkdir -p /oem
sudo blkid ${DEV}p${NUM}
# Note down the UUID
# Configure automatic mounting
sudo vi /etc/fstab
# Add the following to the end of the file (where the UUID needs to be replaced with the real one)
UUID=bbb06fe1-df52-4c7c-b2eb-926b14605fe4 /oem ext4 suid,dev,exec,auto,nouser,async,noatime,nofail 0 0
# Enter the following command to mount the partition
sudo mount /oem

5.4 Restore factory settings

su - root -c 'echo "overlayfs=enable" > /.init_wipedata'
sudo reboot

The next time boot into the ramdisk stage, the userdata partition is formatted and the /var/.init_wipedata file is cleared.

5.5 Debug

If you find that the partition has not changed, there may be an error, check if an error message is printed with the following command:

dmesg | grep initfs