To increase flash lifetime and avoid problems with unexpected power cuts, I run all of my embedded Linux systems from a readonly root filesystem. I find this to be easier than setting up an overlayfs.

This page lists all useful or necessary tweaks I know about. I intend to update it every now and then.

Filesystems

Obviously, /etc/fstab needs to specify that / be mounted readonly. /tmp and /var/log must always be writable in my case, so they each get a tmpfs. Depending on your distribution, this might already be the case -- otherwise, you'll probably need something along the following lines.

PARTUUID=12345678-02  /  ext4  defaults,noatime,ro  0  1
tmpfs  /tmp  tmpfs  auto,noatime  0  0
tmpfs  /var/log  tmpfs  auto,noatime,mode=0755  0  0

Note that tmpfs uses half the available memory as size by default.

Software

systemd

PrivateDevices=yes and PrivateTmp=yes do not work on a readonly filesystem. This affects services such as systemd-timesyncd and can be remedied by running systemctl edit systemd-timesyncd and pasting the following lines.

[Service]
PrivateDevices=no
PrivateTmp=no

You may also need to create tmpfs mountpoints for state files, e.g. for systemd-timesyncd:

tmpfs  /var/lib/systemd/timesync  tmpfs  auto,noatime,size=5m,mode=0755,uid=100,gid=103  0  0

Replace 103/100 with UID/GID of your systemd-timesync user.

logrotate

logrotate keeps state in /var/lib/logrotate and may not work if it can't update it. /etc/fstab:

tmpfs  /var/lib/logrotate  tmpfs  auto,noatime,size=1m,mode=0755  0  0

mpd

mpd won't start if /var/log/mpd doesn't exist, so it must be created after mounting the tmpfs on /var/log. It may also be necessary to create /run/mpd. Create /etc/tmpfiles.d/munin-node.conf with the following content:

d /var/log/mpd 0755 mpd audio - -
d /run/mpd 0755 mpd audio - -

Recent mpd versions also appear to write to /var/lib/mpd periodically:

tmpfs /var/lib/mpd tmpfs auto,noatime,size=10m,mode=0755,uid=112,gid=29 0 0

Replace 112/29 with the UID of mpd and the GID of audio

munin-node

munin-node keeps plugin state in /var/lib/munin-node/plugin-state. If you're lazy, you can symlink that to /tmp, but a tmpfs is a better solution. /etc/fstab:

tmpfs  /var/lib/munin-node/plugin-state  tmpfs  auto,noatime,mode=0755  0  0

Also, munin-node will fail if /var/log/munin-node doesn't exist, so it must be created after mounting the tmpfs on /var/log. /etc/tmpfiles.d/munin-node.conf:

d /var/log/munin 0755 munin adm - -

ntp

ntpd stores state in /var/lib/ntp, so it should be a tmpfs. /etc/fstab:

tmpfs  /var/lib/ntp  tmpfs  auto,noatime,size=1m,mode=0755,uid=106,gid=111  0  0

Replace 106/111 with UID/GID of your ntp user.

Note that this does not apply if you use ntpdate instead of ntp.

pulseaudio

pulseaudio has runtime state in /var/lib and ~/.config/pulse (which contains a symlink to a cookie in /tmp that is updated occasionally). The latter can be remedied by setting XDG_CONFIG_HOME for all pulseaudio processes, though I prefer using tmpfs for both. /etc/fstab:

tmpfs  /var/lib/pulse  tmpfs  auto,noatime,size=1m,uid=110,gid=114,mode=0700  0  0
tmpfs  /home/spotifyd/.config/pulse  tmpfs  auto,noatime,size=1m,uid=1001,gid=1001,mode=0700  0  0

Replace 110/114 with UID/GID of your pulse user and 1001 with UID/GID of each user using pulseaudio.

rsyslog

Some of my readonly systems have a tmpfs on /var/spool/rsyslog, though I'm not sure how important that is. Anyways, the following /etc/fstab line may help:

tmpfs  /var/spool/rsyslog  tmpfs  auto,noatime,mode=0755  0  0

sudo

sudo stores the timestamp of the last successful authentication in /var/lib/sudo, which should be writable. /etc/fstab:

tmpfs  /var/lib/sudo  tmpfs  auto,noatime,size=1m,mode=0700  0  0