Optimizing system performance of flash-based Linux systems

Here are a few tips to significantly improve interactive performance on basic flash-based devices running Linux, such as the Raspberry Pi, other ARM or x86-based development boards, simple netbooks, mini PCs and media boxes, and mobile device such as tablets. These tips apply primarily to Debian and derived distributions such as Raspbian, Ubuntu, Linux Mint, but in general terms apply to most Linux distributions or even Android to some extent.

Typical Linux distributions configured for HDD-based server applications, not optimized for flash-based systems

When the root filesystem used on the device is on a flash memory card or another type of simple flash memory (but not a higher performance recent SSD), the flash storage can quickly become an enormous performance bottleneck, especially given the fact that Linux distributions are typically configured out of the box to continuously access and write to the file system for all kind of logging activity and temporary files used by many applications. While this configuration is suitable for HDD-based PCs or servers, it is not all suitable for a flash-based device.

Possible measures to reduce and optimize flash disk access

As listed below, several types of optimization are possible, each of which can make the system significantly faster, and together they can make a difference of night and day, turning an unusable sluggish system in a fairly quick usable one.

• Reducing system logging activity. Out of the box, Linux distributions tend to be configured with full logging that produces a significant amount of ongoing write access the disk. A lot of data such as kernel messages is often generously logged into two or three different logs. Although logging has its purpose for diagnosing a system problem in a mission-critical system, this does usually not apply to a flash-based device so almost all logging can be safely disabled.
• Using ramdisk filesystems for temporary storage. Most of the time temporary files can easily be stored in RAM, avoiding the significant overhead of storing and modifying a temporary file on flash storage. This involves mounting /tmp and similar directories on a ramdisk (tmpfs), and coaxing applications to store their internal cache or temporary storage directories on a ramdisk. Of course, it is helpful if the the device has a reasonable amount of RAM (512MB is already sufficient for extensive use of ramdisks, while 1GB or more is convenient).
• In general when local cache storage of an application is configurable, an example of which is the web content cache used by a web browser such as Firefox, it can be helpful to eliminate local storage as much as possible (set the size to zero). While obviously more content being kept in RAM by the application of its own volition would be good (this may happen), reloading from the network (internet) is often preferable to the high overhead and bottlenecks caused by the continuous flash disk access for local cache storage.
• The filesystem used, mostly commonly ext4, can be extensively tweaked to provide much better performance on flash storage. Measures taken include using write-back modes with longer "sync" delays instead of ordered data modes with relatively short delays before writing to disk, resulting in much more effective write caching, which can take the edge out of a flash write access bottleneck. Another obvious trick is to eliminate unnecessary bookkeeping such as file access time (use of the " noatime" mount option), and other performance improvements such as forgoing entirely on features such as journalling and huge file support that cater for larger systems and maximal stability on externally powered systems such as PCs. On a battery-powered device, write-back mode/write caching can often be extended without a disproportionate decrease in reliability and stability.

The relevant configuration settings changes are described below. Most of this section has been copied from the recent netbook blog article. Note that superuser priviledges (for example, using sudo) are required for editing most of the configuration files mentioned and any command lines.

Reducing system logging activity

System logging is often configured to be pretty active out of the box with most Linux distributions. Much of the logging that takes place is not a requirement for a (often single-user) flash-based device and can be disabled without consequences. Although kernel logs can be useful, the dmesg command is also able to log kernel messages for the current boot. The system logger is usually rsyslog, the rules for which may be stored in the /etc/rsyslog.d/ directory (for example, in a file named 50-default.conf). Disabling most logging can be accomplished by commenting out the rules by putting a '#' in front of them, for example:

    #auth,authpriv.*        /var/log/auth.log
    #*.*;auth,authpriv.none -/var/log/syslog
    #cron.*                 /var/log/cron.log
    #daemon.*               -/var/log/daemon.log
    #kern.*                 -/var/log/kern.log
    #lpr.*                  -/var/log/lpr.log
    #mail.*                 -/var/log/mail.log
    #user.*                 -/var/log/user.log

Of course, for security purposes or in a multi-user system it might be preferable to keep some of these logs, such as auth.log. Some kernels can be very noisy due to frequent messages or non-fatal errors or warnings which can affect performance when logging is enabled. In this case, it seems reasonable to disable kernel logging via rsyslog because dmesg already produces a similar log.

Using ramdisks (tmpfs) for temporary files

It is quite easy to move directories where temporary files are stored (primarily /tmp) to a ramdisk, and it can make a significant difference in performance. The following lines added to /etc/fstab cause /tmp and /var/tmp to be stored in ramdisks:

    tmpfs    /tmp       tmpfs    defaults    0 0
    tmpfs    /var/tmp   tmpfs    defaults    0 0

Optimizing cache directories

Applications like browsers and window managers that use a disk cache may conform to the XDG Base Directory specification standard. In that case, the environment variable XDG_CACHE_HOME defines the directory where local temporary cache files are stored. By setting this variable to a ramdisk location, it is possible to significantly speed-up the performance of certain browsers that are otherwise affected by heavy writing to the disk-cache on the flash device. This can be accomplished by creating a new file in /etc/profile.d/, for example /etc/profile.d/xdg_cache_home.sh, that will be executed at the start of every shell.

    #!/bin/bash
    export XDG_CACHE_HOME="/dev/shm/.cache"

Note this may not affect the main internet web content cache with certain browsers (such as Firefox), speeding up other types of cached information used by Firefox instead, while it does cause the main internet content cache to be stored on the ramdisk in the case of the lightweight browser Midori. In the case of Firefox, it can be beneficial to reduce the internet cache as much as possible (down to 8MB or zero), since extra network access (as long as it fast enough and not associated with extra cost) likely to be faster than the constant writing to the internet content cache on the flash card that otherwise happens. In the case of Midori, the internet content cache directory on the ramdisk can build up in size, affecting free RAM, which can be fixed by instructing the browser to empty the internet cache on exit.

Optimizing the filesystem (ext4) using write-back mode and other settings

Resources exist on the web on how to improve filesystem performance with ext4. The following line in /etc/fstab illustrates a optimized set of mount options for the ext4 root filesystem that should make a big difference in performance (UUID=nnnn or /dev/sdXn is the partition device used, which depends on the system):

UUID=nnnn / ext4 noatime,journal_async_commit,data=writeback,barrier=0,nobh,errors=remount-ro 0 1

You should also change the physical flag for journal_data_writeback mode, stored in the filesystem itself:

    tune2fs -o journal_data_writeback /dev/sdXn

where sdX is the SD card device and sdXn is the partition where the filesystem is stored. These changes should improve performance a lot, allowing it to reach an acceptable level.

Although these filesystem options have the potential to jeopardize stability and recoverability somewhat in case of system crash or power interruption, when a device is battery-powered the risk is much less.

Disabling X Window System error logging

Finally, the X Window System maintains a logfile called .xsession-errors in your home directory that gets filled with warnings and errors messages from the X server. In some cases this log file can fill up quickly and affect system performance. To disable it, edit the file /etc/X11/Xsession and edit the relevant lines to look like this (somewhere in the middle of the file), In this case the log-generating line has been commented out and replaced with a one that routes messages to /dev/null:

    #exec >> "$ERRFILE" 2>&1
    exec >> /dev/null 2>&1

Conclusion

In summary, the configuration changes above, which are relative to standard configuration settings in typical Linux distributions, can help transform a flash-based Linux system from very slow, continuously stalling behaviour to a reasonably consistent fairly quick response, make it much more usable.

Sources: SmartLogic

Updated November 2, 2014 (spelling).