horneker online

Sysadmin 101: Always Back Up Your System

Introduction

Anytime we work on a computer, whether we write that great term paper, enhance a photograph, or digitize your record/cassette/CD collection, we create data files which are generally stored on your hard disk. (At least it was until the advent of the cloud computing concept.) One would think that these data files are safe stored on a local hard drive. This is, was, and always will be nothing more than a myth. Truth is that data, once saved to disk are available when you next power up your desktop, netbook or laptop. However, what if something unforseen happens, such as accidental erasure of important files, or your hard drive is damaged, or fails completely, or something completely worse like a security breach.

The latter, of course, is very unlikely to happen on Linux/UNIX only if basic security precautions are taken. Thankfully, many distributions have these precautions enabled by default at the time of installation, though the basic security on Mac OS-X is debatable. But this is not the topic of this page.

What I am talking about is the safety of the software and data files on your Linux/UNIX machine. There are many ways to going about backing up of your data and software. Having the original installation DVD helps to restore the Linux distribution in case you need to reinstall Linux.

First, let us discuss backing up of your data. Backing up is defined as making a copy of all files we want to protect from disaster onto a separate physical medium for the purpose of restoring such files if and when needed. On most any Linux machine, user data files are stored in /home/username, where username is the name of the account you are currently logged in. We could backup the entire /home directory, or individual users could backup all files in their respective accounts, resulting in one set of backed up files for each user.

If we are interested in backing up the entire system, we could install Clonezilla, burn a live CD supplied with Clonezilla, and use that and a external hard drive (of the same or greater capacity as the internal hard drive) to backup the entire contents of your hard drive.

Many of us just want to backup our data files. In the early days of computing (late 1960s to the 1990s), we typically used magnetic tape or floppy disks to create data backups. These days, blank CD/DVD discs, external hard drives, memory cards, and USB flash drives are the common medium in which to backup our data.

This page covers backing up of data on one Linux/UNIX machine, typically that which is used in a home, on the road (as in a laptop or netbook), or in a small business. Backing up of data on multiple machines throughout a network is beyond the scope of this page, and requires additional (and somewhat complicated) procedures to accomplish the task.

Blank CD/DVD Media

Backing up to CD/DVD media (or even Blu-ray discs) is a inexpensive and somewhat permanent way to preserve your data. You will need a CD burning program such as K3B or Brasero to accomplish this task, and a compatible CD/DVD writer. Fortunately, most DVD writers will work with Linux, be it internal or external, so finding compatible hardware should not be an issue.

The medium on the other hand can be problematic. Not all medium is compatible with all CD/DVD/Blu-Ray writers. The following is a table of the disc types and their capacities:

Disc Type Capacity Notes
CD-R 700MB Available in CD-R Data and CD-R Audio, with the difference being that CD-R Audio was optimized for recording audio CDs. These discs can only be recorded once, and never recorded on again. This type of disc is ideal for small (700MB or less) archives of photographs and documents
CD-RW 700MB These type of discs are reusable. Formatting of these discs is required if they are to be reused. Note: Some older Sony CD-RWs have a capacity of 650MB rather than 700MB.
DVD-R 4.7GB These discs can only be recorded once in a single session. This means that once you have recorded on them, the discs cannot be reused. This is ideal for full and long term backups of your data.
DVD+R 4.7GB These discs can be recorded more than once, but each session is appended to the end of the previous session on the disc. For these types of discs, I recommend backing up as close to 4.7GB as possible of your data to make the most of these discs.
DVD-RW 4.7GB Same as DVD-R discs, except that these are reusable. You need to reformat these discs to reuse them.
DVD+RW 4.7GB These are the most versatile of the DVD discs. You can reuse them, and there is no need to reformat these discs. Simply rewrite over them.
Blu-Ray 50GB This type of optical CD is large enough to hold the PCLinuxOS repository (a whopping 50GB capacity).

There is a Double Layer variant of the DVD-R and DVD+R. I have not been able to successfully write to these discs, even though my DVD writer is supposed to support these type of discs.

Backing up data using blank CD/DVD discs requires some planning on not only how many discs to use for the backup, but how to organize the data being backed up. For example, you could use two blank DVDs for storing video projects, one blank DVD for photographs, one blank DVD for documents, and one blank DVD for music stored on your computer.

USB Mass Storage Devices

A more common medium for backing up of data is the use of USB mass storage devices. Linux treats these devices as if they were just another disk. To backup files to these devices, simply mount, copy the files to be backed up, then unmount when you are done. This depends on the desktop environment you are using (such as KDE, GNOME, MATE, LXDE or XFCE) and whether you are using a file manager or not.

USB flash drives are the most portable and easiest to use for data backup. Simply plug it in and Linux will automatically mount the flash drive (depending on the desktop environment you are running). Flash drives range in capacity from 2GB to 128GB (though older flash drives have much smaller storage capacities).

Memory cards such as SD/MMC (all variants), Memory Stick(TM) (all variants), and Compact Flash, are commonly used in devices such as digital cameras, media players, portable gaming consoles, cellular phones (smartphones as well as the basic models), and the Raspberry PI . Memory cards can also be used for backup storage as well as Linux installation (especially on netbooks). As with USB flash drives, capacities vary from 32MB (for the oldest memory cards) to 256GB (which is enough to backup some hard drives) available.

External hard drives are the preferred devices for backing up of data. They are quite expensive when compared to USB flash drives and memory cards, but they are designed to be used to backup your entire hard drive. Capacities (as of this writing) range from 256GB to 2TB. This is the preferred medium for use with the Clonezilla product to backup your entire hard drive.

Other devices such as media players contain flash memory and can be used as backup devices. However, I do not recommend doing so if you plan to use that device for which it was intended.

One thing that is common with USB mass storage devices is that they generally come formatted with the VFAT filesystem. VFAT stands for Virtual File Allocation Table, and is called FAT32 in the Windows world. File Allocation Tables are a small portion of the disk set aside to keep track of what files are where on the disk and how much space they take up. This file system was introduced with the original IBM Personal Computer back in 1981, and has been in use ever since. In fact, USB mass storage devices are the only reason that FAT is still in use today.

USB 3 Considerations

USB 3.0 is the latest specification that promises data transfers that much faster than USB 2.0. To get that kind of data transfer speed, you must have a USB 3.x controller either built-in to your machine, or a separate controller installed on your machine.

Thankfully, USB 3.x is backwards compatible with USB 2.0 devices. The downside here is that data transfers are limited to USB 2.0 speeds (about 480 megabits per second versus 5 gigabits per second for USB 3.0, and 10 gigabits per second for USB 3.1).

Also, a recent Linux kernel (at least a 4.0 series) is required to use USB 3.0 and 3.1 devices under Linux. But the kernel is only half the equation for getting USB 3.x devices to work under Linux.

USB 3.0 was designed to work with desktops and laptops built in the 2010s. So, if you have an older laptop or desktop, there is no guarantee your machine will be able to accommodate a USB 3.0 or 3.1 controller. In this case, you may have to settle for USB 2.0 speeds.

That brings us to the topic of USB 3.0 storage devices. Before purchasing a USB 3.0 flash drive or external hard drive, make sure it can work with USB 2.0 controllers, or you are going to have serious issues with your desktop or laptop using the flash drive or external hard drive.

I have seen some newer flash drives available that are USB 3.0 only. Avoid these unless you have a USB 3.0 controller installed on your machine.

Cloud Services are not recommended

Thanks to the advent of cloud computing (and Web 2.0 in general), there is another means for backing up your data. Services like Carbonite allow you to backup your data onto remote servers (for a nominal fee, of course).

While this may keep your data safe and accessible, the security of your data comes into question. How can we be really sure that the data you store on remote servers is safe from cracker attacks, or from prying eyes? Companies that provide such services can only promise such protection such as encryption of your data. There is still the possibility that your data is not truly safe, especially at a time when surveillance issues are still hot topics.

Because of this, I still recommend using physical medium to backup the data files on your hard drive.

Organizing Your Data

Making backups of your data can be easier if you have your data files organized in your home directory (or folder if you are used to Windows and Mac OS). By organizing your data files, you can easily select which files to backup simply by specifying the directory they are stored in. (Another reason for doing this is that some applications such as DigiKam, Banshee, and Amarok expect their data files to be located in subdirectories such as /home/username/Pictures and /home/username/Music, where username is the name of the account you logged into.

Many distributions such as OpenSuSE (and my derivative), Ubuntu, PCLinuxOS, and Fedora already setup some of these directories for you at installation time.

However, some distributions such as Slackware only setup a home directory. It is up to you to setup the structure for organizing your data. The following are typical directories on your user account and their intended purposes

Desktop: This directory contains files created and used by desktop environments such as KDE, XFCE, LXDE and GNOME.

Documents: This is where documents you create and edit are normally stored.

Downloads: Anything downloaded from a web browser such as Firefox and Chrome is stored here.

GNUstep: This directory is used by WindowMaker and AfterStep window managers to store their configuration and resource files.

Music: This directory contains digitized music files and is the default directory for Amarok, Banshee, Rhythmbox, and other software media players. Subdirectories here generally are named to represent albums that have been digitized.

Pictures: This directory is where digital photographs are stored, and is the directory expected to be present with DigiKam and Shotwell.

Videos: This directory is where digital videos are stored, and is used by MPlayer, SMPlayer, UMPlayer, Xine, and Totem

How Much and When

Here, we need to make a plan on what type of backup we are doing, and how often to perform backups.

Full Disk Backup: This is where we need to copy the contents of the entire hard disk. I recommend Clonezilla and a external hard drive of at least the capacity of the hard drive on your computer for this task. After installing Clonezilla, burn the liveCD image that comes with Clonezilla and reboot your system with that and your external hard drive attached to backup your hard drive.

Full Data Backup: For this type of backup, you simply make a copy of the contents of /home onto blank CD/DVD discs, USB flash drives, memory cards, or to an external hard drive.

Full Home Directory Backup: Use this option if you are the only user on your Linux system, or if you want individual users to be responsible for their own data backups.

Incremental Backups: This option supplements the full backup options by providing updates to the full backup set. To restore data if needed, you will need to restore the full backup first, then the incremental backups.

Software for Making Backups

Though software is available dedicated to making backups of your data, your Linux distribution contains all the tools you need to accomplish this task (unless you are planning to backup data and software).

Applications such as K3B and Brasero can be used to create data backups to CD/DVD/Blu-Ray discs. Making backups to USB storage devices is much easier. Simply plugin and mount the device, copy the files to be backed up, then unmount the device.

Finding the Device Name for USB Storage Devices

Let us say we want to use a flash drive for a backup device. If you are running KDE, GNOME, XFCE or LXDE, simply plugin the flash drive and an icon representing the flash drive will appear on the screen. All you need to do is double click on the icon to mount the flash drive and open a file manager window. Then double click on the icon labelled Home to get the contents of your home directory. To backup these contents, simply select all icons (Hold down Control, then press A to select all icons), then using your mouse, drag these icons to the window representing your flash drive. If a popup menu appears, select Copy and watch the action on your screen as your home directory files are being copied to your flash drive.

But what if you are using another desktop environment such as WindowMaker, or are in the console (text) mode. You will need to find the device name for your flash drive when you plug it in. Plug in the flash drive and open a terminal window (if you are using a desktop environment). You will need to login as root for the next step.

With the terminal window open type su -c "dmesg", press Return, then type in the administration password (i.e. the password for the root user) and press Return. If you are using Ubuntu or a distribution that uses sudo rather than su, then you would type in sudo dmesg, then the administrator password.

You should then see something like this:

 	[17127.770150] usb 2-1: new high-speed USB device number 7 using ehci-pci
	[17127.886832] usb 2-1: New USB device found, idVendor=0c76, idProduct=0005
	[17127.886842] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
	[17127.886850] usb 2-1: Product: USB Flash Disk  
	[17127.886857] usb 2-1: Manufacturer: 3SYSTEM 
	[17127.886863] usb 2-1: SerialNumber: 00EA0109207BF9B1A4282BE4
	[17127.887432] usb-storage 2-1:1.0: USB Mass Storage device detected
	[17127.887606] scsi11 : usb-storage 2-1:1.0
	[17128.889890] scsi 11:0:0:0: Direct-Access     3SYSTEM  USB Flash Disk   1.00 PQ: 0 ANSI: 2
	[17128.891981] sd 11:0:0:0: [sdc] 8002864 512-byte logical blocks: (4.09 GB/3.81 GiB)
	[17128.892825] sd 11:0:0:0: [sdc] Write Protect is off
	[17128.892835] sd 11:0:0:0: [sdc] Mode Sense: 03 00 00 00
	[17128.893455] sd 11:0:0:0: [sdc] No Caching mode page found
	[17128.893464] sd 11:0:0:0: [sdc] Assuming drive cache: write through
	[17128.896572] sd 11:0:0:0: [sdc] No Caching mode page found
	[17128.896582] sd 11:0:0:0: [sdc] Assuming drive cache: write through
	[17128.897493]  sdc: sdc1
	[17128.900227] sd 11:0:0:0: [sdc] No Caching mode page found
	[17128.900231] sd 11:0:0:0: [sdc] Assuming drive cache: write through
	[17128.900235] sd 11:0:0:0: [sdc] Attached SCSI removable disk

 	

This example represents a DaneElec 4GB flash drive that I use to backup my photographs, with Solid State System (of Zhubei City, Taiwan) being the actual manufacturer of the flash drive. Here, the device was detected by Linux and assigned the device name /dev/sdc. On my laptop, /dev/sdb represents the built-in SD card reader and /dev/sda represents the hard drive. Note that the flash drive has only one partition, formatted with the VFAT filesystem, for compatibility with other Linux/UNIX systems, Mac OS-X, and of course Windows. This partition is given the name /dev/sdc1. /dev/sdc1 is the name that we want to use to mount the flash drive before copying files. /dev/sdc represents the entire flash drive.

Mounting the Flash Drive

There are two directories in UNIX world that we can use to mount the flash drive. They are /mnt and /media. The latter was intended to be used for removable devices, such as flash drives, while the former was intended to be used for hard drives and devices that are constantly in use throughout a Linux session, such as a second hard drive (commonly found on servers and desktops), or a connection to a server within the same internal network or intranet.

I recommend using /media to mount the flash drive. Mounting the flash drive requires administrator access, so we must login as root for the next step. This is done by typing su or sudo -i, then entering the administrator password.

Once logged in as root, type ls -l /media. We should see something like this:

	bash-4.2# ls -l /media
	total 60
	-rw-r--r-- 1 root root  500 Sep 25  2006 README
	lrwxrwxrwx 1 root root   11 Jun 18 13:23 cdrecorder -> cdrecorder0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 cdrecorder0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 cdrecorder1
	lrwxrwxrwx 1 root root    6 Jun 18 13:23 cdrom -> cdrom0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 cdrom0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 cdrom1
	lrwxrwxrwx 1 root root    4 Jun 18 13:23 dvd -> dvd0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 dvd0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 dvd1
	lrwxrwxrwx 1 root root    7 Jun 18 13:23 floppy -> floppy0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 floppy0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 floppy1
	lrwxrwxrwx 1 root root    3 Jun 18 13:23 hd -> hd0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 hd0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 hd1
	lrwxrwxrwx 1 root root    7 Jun 18 13:23 memory -> memory0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 memory0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 memory1
	lrwxrwxrwx 1 root root    4 Jun 18 13:23 zip -> zip0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 zip0
	drwxr-xr-x 2 root root 4096 Sep 25  2006 zip1
	bash-4.2# 

	

This is output from a Slackware 14.1 system. The total 60 means that 60 blocks on the hard drive were used to store the contents of /media using the ext4 filesystem.

For a flash drive, /media/memory would be an appropriate choice of a place to mount the flash drive for copying files. We could also use /media/floppy (intended for floppy drives for those of us who remember what floppy drives were) or /media/zip (intended for the defunct line of IOmega Zip drives of yesteryear) as well. For this example, let us use /media/memory.

Now that we have a device name and a mount point, all we need to do is mount the flash drive. We accomplish this by typing:

mount -t vfat /dev/sdc1 /media/memory

Now, Make the Backup

This example was done with a flash drive. However, we could have been accomplished using any USB storage device. Assuming that we have sufficient storage space on the flash drive, we go back into our home directory. In the command line environment, the flash drive can only be written to in administrator mode. To get back to your home directory, type cd /home/username where username is the account name you logged into.

Now that we are in your home directory (and still in administrator mode), we type the following:

cp -r * /media/memory

The -r means to copy the contents of any subdirectories that exist within your home directory. The * means copy all files.

Unmount the Flash Drive to Finish

After we are cone making our backups, the final step to close out this operation is to unmount the flash drive. Unmounting means detaching the contents of the flash drive from the directory structure so the files will be preserved on the flash drive. As we are still in administrator mode, and are in your home directory, we simply unmount the flash drive by typing:

umount /dev/sdc1

Alternately, we could accomplish the same with:

umount /media/memory

Either way, the umount (for unmount) command tells Linux to close any file operations on the flash drive and disconnect the flash drive from the directory structure.

If everything went OK, then you have successfully backed up your home directory.