Using mdadm to create a software RAID 1 array on Linux

Linux Software Raid 1 Setup

RAID 1 is a hard disk configuration where the contents from one hard disk are mirrored onto another. This provides the user with some redundancy in case a disk fails. On your Linux system, the two hard drives are represented as a single file system. But in the background, making changes to your files is actually writing the changes to two disks at the same time. You can also add more than two disks to the configuration, as long as you keep the number even. Otherwise, something like RAID 5 will be more suitable.

There are many ways to configure a RAID setup. One of the easiest and most accessible ways is through the mdadm software package, which can be installed and used on any major Linux distribution. This is easier than some other RAID setups, since it doesn’t require any special hardware (like a RAID controller) and isn’t that hard to configure.

In this guide, we’ll go through the step by step instructions to install and setup mdadm on Linux, and create a RAID 1 configuration for two hard disks. Our example scenario will consist of two empty hard disks that are each 10 GB in size. This is in addition to our main hard disk, which is just used for the operating system.

WARNING
Strictly speaking, RAID 1 is not a proper backup solution. It does provide some protection from disk failure, but what if you accidentally delete a file or a virus corrupts multiple files? Those undesirable changes are instantly written to both disks. RAID 1 provides high availability, but you shouldn’t use it as your only backup solution.

In this tutorial you will learn:

  • How to install mdadm on major Linux distros
  • How to partition hard disks for RAID setup
  • How to create a new RAID device in mdadm and mount it
  • How to keep the RAID array mount persistent
Using mdadm to create a software RAID 1 array on Linux

Using mdadm to create a software RAID 1 array on Linux

Read more

s3cmd – Backup directly to Amazon S3 storage using STDOUT

From time to time you may want to backup your files directly to Amazon S3 storage without the middle step of saving your backup compressed file to a local disk. This is especially important if you have a limited disk space on your local drive/server. The below commands assume that you have already installed and configured s3cmd tool on your server. The bucket name we are going to use for our examples is called backup. Let’s start with a regular backup creating a local file.

$ tar cPf /tmp/lubos.tar /home/lubos
$ s3cmd put /tmp/lubos.tar s3://backup/lubos.tar

Read more

How to backup and restore permissions of the entire directory on Linux

The following two commands getfacl and setfacl are very handy tools as they allow Linux administrators to take a snapshot of any current permissions settings of any directory and if needed re-apply those permissions back recursively. Let’s have a look at the following example:

$ tree -p
.
├── [dr---w----]  dir1
│   └── [drwxr-xr-x]  dir2
│       ├── [dr--r-xrw-]  dir3
│       └── [---x--x--x]  file1
├── [drwxr-xr-x]  dir4
│   └── [-rw-r--r--]  file3
└── [-rwxrwxrwx]  file2

4 directories, 3 files

Read more

whereis rsnapshot-diff

Guide to rsnapshot and incremental backups on Linux

Introduction

rsnapshot is a backup tool written in Perl that utilizes rsync as its back-end. rsnapshot allows users to create customized incremental backup solutions. This article will discuss the following: the benefits of an incremental backup solution, rsnapshot’s installation, its configuration, and usage examples.

Back-it up!

I was recently discussing with a colleague the benefits of backing up your data. My colleague was telling me how one of her customers had recently lost a rather lengthy article that they had been working on. I decided that this may be a good chance to experiment with my netbook and rsnapshot. For this tutorial, I’ll assume you have have 2 pieces of hardware: your host computer, and your destination equipment. I’ll be using an external hard drive for the majority of this post. However, I will briefly cover usage for backing up files over a LAN.

Backing up your data should not be the question to ask but rather how should I backup my stuff? What’s the best way? Well there are many different backup pathways you can take, including block level (dd, partimage), partition level (RAID and all its variations), file level (rsyncand its children applications). I’ll discuss two types of backups in the context of file-based backups.

Normal backups, or full backups, are self explanatory. Normal backups are one way of backing up ALL your files every time you perform a backup. One issue with utilizing a multiple normal backup scheme is that a normal backup takes up a considerable amount of space. For example, if you perform a full backup of a 250gig hard drive at 20% capacity, everyday for just one week (assuming that the amount of data does not fluctuate) will mean that you already have used 350gigs for only one week’s worth of backups. As you can see, that is not feasible in the long run. The other method that I prefer is the incremental backup method. An incremental backup consists of one full backup and then performing additional backups. These additional backups will only backup files that have changed since the last backup. Instead of backing up your entire hard drive, only the specific files that have changed since the last backup are backed up. As you can probably imagine this is a much more efficient process. One tool that does this on *nix is rsnapshot.

Read more

How to create incremental backups using rsync on Linux

How to create incremental backups using rsync on Linux

In previous articles, we already talked about how we can perform local and remote backups using rsync and how to setup the rsync daemon. In this tutorial we will learn a very useful technique we can use to perform incremental backups, and schedule them using the good old cron.

In this tutorial you will learn:

  • The difference between hard and symbolic links
  • What is an incremental backup
  • How the rsync –link-dest option works
  • How to create incremental backups using rsync
  • How to schedule backups using cron

Read more

Rsync Examples

Examples on how to use Rsync for local and remote data backups and synchonizations

Rsync is a very useful tool which allows Linux system administrators synchronize data locally or with a remote filesystem via the ssh protocol or by using the rsync daemon. Using rsync is more convenient than simply copying data, because it is able to spot and synchronize only the differences between a source and a destination. The program has options to preserve standard and extended filesystem permissions, compress the data during transfers and more. We will see the most used ones in this guide.

In this tutorial you will learn:

  • How to use rsync to syncronize data
  • How to use rsync with a remote filesystem via ssh
  • How to use rsync with a remote filesystem via the rsync daemon
  • How to exclude files from the synchronization

Read more

How to setup the rsync daemon on Linux

How to setup the rsync daemon on Linux

In a previous article we saw some basic examples of how to use rsync on Linux to transfer data efficiently. As we saw, to synchronize data with a remote machine we can use both a remote shell as  ssh or the rsync daemon. In this article we will focus on the latter option, and we will see how to install and configure rsyncd on some of the most used Linux distributions.

In this tutorial you will learn:

  • How to install and configure the rsync daemon

Read more

How to Create Backups with Fsarchiver on Linux

How to Create Backups with Fsarchiver on Linux

Fsarchiver is a free software utility that let us create file-level backups of one or multiple filesystems in a single archive. One big advantage of this kind of backup is that we can restore it on a filesystem smaller than the original one (but of course large enough to contain all the files); this is usually impossible when performing block-level backups, using tools like partclone or dd. In this article we will learn how to install and use the application and its main features.

Read more