#6 Linux LVM, RAID, caching 2015

In the initial LVM/RAID notes a very long time ago (in a galaxy far far away) we used the classic Linux Multiple Device software RAID setup.

However, in the meantime it is possible to setup some RAID levels directly within the LVM management to simplify and streamline the experience:

Native LVM RAID

Nowadays, we can intead let LVM manage all this by simply adding all single devices as physical volumes and combine them to a volume-group:

What is new is, that we can now set up the RAID directly inside the LVM:

Of course various RAID levels are available, too (but may require additional parametners). This not only simplifies the setup, it also makes the whole setup easier to manage, and to easily expand in the future.

Additional you can combine this with all the latest and greatest thin-pools, snapshots, and whatnot that may be great options for virutalization.

Adding SSD caching

One of the greatest and latest options is adding a cache to the logical volume, usually to improve the performance of large spinning disks. Especially seek times and thus operations per second are much higher for solid state storage. So by adding small, high performance SSD you can now enable caching for your LVM setup:

and you should get a confirmation like:

Behind the scene this command will create two volumes for the cache- and meta-data. However, latest lvm tools automatically manage this complexicty for you.

You can also ask for some basic statics about the utilization and hits vs. misses, like:

Living a dangerous life on the fast lane

By default LVM2 defaults to a saver, but slower write-through cache mode, which considers a write complete only when it has been stored in both the cache pool and on the origin LV. If your goal is to maximize performance by also hiding write latency by utilizing SSDs, you might want to setup write-back cache mode via:

which considers a write complete as soon as it is stored in the cache pool. To improve resilience, you could create the cache on a multi device, mirrosed LV.

Added bonus: Crypt the whole thing!1!

There are many reasons why you may want to add some crpyto into the mix. May it be simply your accouning and tax backup, your trade secret, running on an external root server, or all your business and prive emails and documents. With Linux, device mapper and all it's infrastrucutre it is as easy as:

and then setting everything up on top of /dev/mapper/crypto.

More notes

Create RAID5 with specifc devices, e.g. not on a ssd only intended as cache:

Extend just created RAID5 volume to 2TB:

Followed by resizing the underlying file-system by something like:

Uncaching (removing) a cache

One more thing: hot-adding RAID5 devices

So you need more storage and want to add another storage device to your RAID(5) array? It appears to be as simple as:

On success you should see your RAID re-syncing with a message simillar to:
md: reshape of RAID array mdX

Last but not least: replacing drives

So you got a new drive and want to migrate data before an old device failes, or simply accidently or automatically allocated space on a wrong, e.g. cache NVMe SSD? You can simply migrate data among physical volumes like:

External links

• Linux LVM
• bcache and/vs. LVM cache

The Author

René Rebe studied computer science and digital media science at the University of Applied Sciences of Berlin, Germany. He is the founder of the T2 Linux SDE, ExactImage, and contributer to various projects in the open source ecosystem for more than 20 years, now. He also founded the Berlin-based software company ExactCODE GmbH. A company dedicated to reliable software solutions that just work, every day.