Post Reply 
 
Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
So you want to run a RAID array...
Author Message
Defiant Offline
Posting Leader
*****

Posts: 4,654
Joined: Nov 2010
Reputation: 64
Thanks: 59
Given 594 thanks in 458 posts
       
Post: #1
So you want to run a RAID array...
Lately I've noticed a lot of people have had questions and shown interest in the ability to run a RAID array. These days with most motherboards having onboard RAID ability in some fashion or another it is an easy option to get into for only the cost of another hard drive making it MUCH cheaper then the cost of a proper PCI/PCI-Express controller plus hard drives/solid state drives. RAID actually stands for Redundant Array of Inexpensive Disks or Redundant Array of Independent Disks depending on who you ask or where you look it up, both are true actually.

However, onboard controllers DO offload CPU cycles to the motherboard's processor which does make them slower than their dedicated equivalents. They also lack certain special recovery features such as BBUs (Battery Backup Units) and expandable cache:

RAM style expandable cache:
(You may also notice an ethernet port on this model, some have this function for remote management of the array and NTP sync)
[Image: thb54450.jpg]

Battery Backup Unit (BBU) provides just enough power for the RAID controller to continue writing all remaining data in its cache to the connected volume which greatly reduces the chances of a corrupted array in a power failure (UPS devices can fail too Wink ):
[Image: MegaRAID320-2.jpg]

When you get into enterprise RAID controllers such as this you have the ability (on the really high end ones) to actually span entire arrays across multiple controllers (For example if you wanted a 20+ drive array you would probably use several controllers).

Alright, now you have an idea of some of the technologies used in RAID lets actually talk about the types of RAID. Keep in mind that while any drive (solid state drive or hard drive) is compatible with RAID, you should only use matching drives (that is of same size/manufacturer/model/down to revision if possible) to avoid corrupted arrays, inconsistent speeds, false-positive drive death, and many other nasty things. I do not ever recommend using non-matched drives in any array, this is asking for trouble and it will catch up with you. Something to keep in mind is that most hard drives are not "aware" they are in an array, they are just happy to receive ones and zeros and write and read them as ordered. Some enterprise drives such as the Western Digital RE series have RAID features but for the most part it is best to think of them as "buckets" which are part of a greater "bucket".

"Common" RAID levels:

RAID 0: This one is the most common in home use. Basically you need a minimum of 2 drives for this and the array size will be all of the drives total size combined, for example with two 500 GB drives the array with be 1000 GB or 1 TB, with three 500 GB drives the array will be 1500 GB or 1.5 TB. The controller treats this in the same manor as SLI for video cards and "splits" the work between the 2, this allows for a boost in speed though it will not be twice as fast due to controller overhead. You can have more than 2 drives in this type of array but after adding a third drive the speeds start to increase less and less, this type of RAID is best kept under 4 drives IMO. The reason I say this is because there are better options for more drives, and with RAID 0 there is zero redundancy, if any drive fails the whole array is toast. Use only with proper backups!

RAID 1: This is a common array in workstations at a business in my experience. This provides no speed increase and in some cases actually decreases the speeds, however this allows for all but one of the drives in this type of array to die without data loss. The minimum drives needed for this are 2 and the array size will be the size of one of the drives, for example with 2 or 3 500 GB drives the size will be 500 GB. There is really no point to having more than 2 drives in this configuration unless you are very paranoid about data loss, but this is not a recommended backup strategy as it is still vulnerable to good old viruses and other data corruption. This is really only a method of preventing downtime due to hardware fault. ALWAYS have a proper backup complimenting this type of setup, usually running this array means you deal with important work in production of some sort. If a drive in this array fails, replace it ASAP as with 1 drive gone your redundancy is gone!

RAID 5: But Defiant, I want redundancy AND speed. Well then, RAID 5 is usually for you! Raid 5 uses something called a distributed parity block which makes it different from RAID 1 and RAID 0. This allows any one drive in a RAID 5 array to die without killing the array and requires a minimum of 3 disks and you will lose the space of 1 disk, for example with three 500 GB drives you will have 1000 GB or 1 TB, with four 500 GB drives you will have 1500 GB or 1.5 TB. It simply rebuilds the missing drive from the handy distributed parity! With this array type you get speeds a little less than RAID 0 and redundancy a little less than RAID 1, neat eh? However there is a downside.. RAID 5 becomes very slow when operating with a dead drive, the whole array will run very slow until a new drive is swapped in and fully rebuilt. This can be helped with the addition of something called a "hot spare" which is a drive inside the array which is powered and ready but dormant until activated by the RAID controller in the event of another drives death. This drive will immediately start being rebuilt from the distributed parity, once the dead drive is replaced it becomes the new hot spare. This RAID level is also where onboard RAID controllers start to become a bad idea. The distributed parity calculations are very processor heavy on the go and without a dedicated card you will start to get lower IO (Input Output) speed.
Side note: You may have also heard the term "cold spare", this is different from a hot spare because this is a drive which is not powered up and still needs to be attached to the array. Drives like this are usually sitting on a shelf near the server/computer with the array.

Raid 6: Basically the same as RAID 5 except with dual distributed parity (damn that sounds fancy eh?). This essentially means you can have 2 drives die before the array loses all redundancy. This is better for when using more drives such as 8 to 10. You need a minimum of 4 drives to run this type of array and you will lose the space of 2 drives. For example with four 500 GB drives you will have a 1000GB array or 1 TB, with five 500 GB drives you would have 1500 GB or 1.5 TB. As with RAID 5 but even more so it is recommended to use a dedicated controller especially due to the dual distributed parity blocks.

"Nested" RAID levels (a.k.a RAID inside a RAID):

RAID 0+1: This is most common in an enterprise/business environment. I won't go too much into this. Basically you can have a minimum of 3 drives for this but I would really recommend using 4. You will lose half the space of the array. What this does is creates two RAID 0 arrays and then puts them together in a RAID 1. Think of it as RAIDing together two really fast and large buckets into a bigger bucket, there is no cabling difference, its all done on the controller. Redundancy is weird as you can lose ALL the drives from a single side of the array without array loss, but if you lose 1 drive from each of the initial RAID 0s the array will be destroyed. Speed is what you would expect, it would be as fast as one of the initial RAID 0 arrays, this is entirely based on what each initial RAID 0 is composed of drive-wise and how many.

RAID 10 (a.k.a RAID 1+0): Similar to RAID 0+1 except the drive nesting order is reversed. This allows more redundancy while reducing speed ability. Drive space in this type of array is only the size of two of the array's drives. For example if the array is six 500 GB drives the total array size would be 1000 GB or 1 TB.

Note: With Nested RAID levels you basically take one RAID level and then RAID it with another. This type of RAID level starts to be used with lots of drives, over 10 usually.

Other Array Types:

JBOD (Just a Bunch of Disks): Exactly what it sounds like. What this does is "attach" several disks together one after the other in a "software line". Because of this you may (and only this time) have different drives in the array. This type of array has no redundancy, so if any drive dies the whole array is nerfed.

Software RAID vs Hardware RAID:

Software RAID: This is the RAID type that is built into an Operating System or does not have dedicated processing, I am going to use Window as the example here, but Linux has its own version of this too which is more advanced. Any RAID level you use here will be of the same concepts described above (Spanned = JBOD, Striped = RAID 0, Mirrored = RAID 1, RAID 5 = Obviously RAID 5). The problem with this is that it is all run through software which slows things down a lot. But the major problem is that this array is linked to the Windows install. Therefore if your Windows install gets bricked, so does your array, reinstalling Windows will not recover the array.
[Image: Windows_raid-1.png]

Note: Cheap dedicated RAID controllers (under $200 usually) are also technically considered software RAID because they offload calculations to the motherboard's processor in a similar fashion to all onboard RAID. Ultra cheap PCI/PCI-E network cards also do this which defeats the purpose of having them unless you just want an extra port. This is a major factor when buying any cheap dedicated PCI/PCI-E device.

Hardware RAID: Little explanation is needed here as any good RAID controller uses this which means it has an onboard processor and cache. It will be faster but cost more.


Personal RAID Arrays:

Now for some insight to what I personally run...

My motherboard is an Asus P6T Deluxe with the Intel ICH10R chipset which has one of the best RAID controller for an onboard as far as I'm concerned. I am running two RAID 0 arrays off of this setup.

My primary gamer (I have 2) boots off of 2 Corsair F60s in RAID 0:
[Image: SSDRAID.png]

For game storage (Mostly Steam) I use my old boot array which is two Western Digital Black 1 TB SATA2 (the older one) drives in RAID 0:
[Image: HDDRAID.png]


Thats everything I can think of for the moment that would most relevant, I will add more later or if you guys want to know anything else ask below. If an admin could make this a sticky that would be awesome, I think this could help a bunch of people.

Rigs:
Defiant (Gamer)
Intrepid (HTPC)
Galaxy (Server)
(This post was last modified: 07-26-2011 11:30 PM by Defiant.)
07-26-2011 11:14 PM
Visit this user's website Find all posts by this user Quote this message in a reply
 Thanks given by: sprogis , Starfire , Johnny "Draconis" Su , Mahloot
Mystryman Offline
INTERNATIONAL MAN OF MYSTRY
*****

Posts: 4,349
Joined: Dec 2010
Reputation: 42
Thanks: 454
Given 267 thanks in 236 posts
     
Post: #2
RE: So you want to run a RAID array...
Gahh, Ill read this when I get home.

ALL HAIL DISTWROUGHT!
BASED DISTWROUGHT
PRAISE, DISTWROUGHT SUPREME LEADER

#DISTWROUGHTLYFE
07-27-2011 09:02 AM
Find all posts by this user Quote this message in a reply
Steven S Offline
Forum Helper
*****

Posts: 1,069
Joined: Dec 2010
Reputation: 10
Thanks: 8
Given 84 thanks in 70 posts
 
Post: #3
RE: So you want to run a RAID array...
keep in mind different software RAID will have different performance. here's the P67 RAID controller in action:

2x intel 320 120GB:
[Image: intel320raid032katto.png]

1TB caviar black SATA3 and 1TB barracuda SATA3:
[Image: 1tbhddraid0.png]

the P67 RAID controller clearly shows a preference towards small file read speed. at first i thought the vastly different write vs read speeds on the SSD array is because of the SSDs themselves, but then, the mixed HDD array shows the same behavior, though not as pronounced.

i believe the answer is a combination of both factors. the intel 320 series and the X25-M G2 before it are both tuned to have stronger read performance, especially for 4KB random reads. in fact, if memory serves me correctly, the X25-M G2 has a sequential write performance lower than a 2TB caviar green, but could run circles around most other SSDs for small random reads.

i'm a bit surprised that the HDD array shows the same behavior, though not as pronounced. seeing that the only common factor between them is the controller, i believe the P67 RAID controller simply shows a preference over small file read performance (and possibly latency) than write. i wonder if it is possible to change that for the HDD array, since it is used as my scratch disk for FRAPS recording, and i need as much sequential write as possible for that.

Intel i7 2600K @ 4.5Ghz
16GB G.Skill Ripjaws Z 1600Mhz
Asrock P67 Fatal1ty Professional
MSI HD 7970 Lightning @ 1125/1600 MHz
Crucial M4 256GB SSD
3x Seagate Barracuda 1TB 7200.12 HDDs (RAID0)
Seagate Barracuda Green 2TB HDD
Silverstone Fortress FT02
07-27-2011 12:40 PM
Find all posts by this user Quote this message in a reply
Peter M Dodge Offline
Posts Dominator
*****

Posts: 5,010
Joined: Dec 2010
Reputation: 29
Thanks: 12
Given 138 thanks in 127 posts
     
Post: #4
RE: So you want to run a RAID array...
CC should carry some more professional grade RAID cards. I know I'd buy them, I have to go elsewhere ATM.

[Image: currentbuild.png]
07-27-2011 01:28 PM
Visit this user's website Find all posts by this user Quote this message in a reply
Defiant Offline
Posting Leader
*****

Posts: 4,654
Joined: Nov 2010
Reputation: 64
Thanks: 59
Given 594 thanks in 458 posts
       
Post: #5
RE: So you want to run a RAID array...
(07-27-2011 12:40 PM)Steven S Wrote:  the P67 RAID controller clearly shows a preference towards small file read speed. at first i thought the vastly different write vs read speeds on the SSD array is because of the SSDs themselves, but then, the mixed HDD array shows the same behavior, though not as pronounced.

I have a better explanation. RAID stripe size. What did you set it to? The smaller you set the RAID stripe size to the better it is for operating systems due to lots of small files, larger stripe sizes are great for when dealing with storage arrays that will store larger files (media or otherwise). Basically the stripe size determines how much data is written to each drive at a time, with a stripe of 128 KB a 1 GB file will have a 128 KB piece on "Drive A", then the next 128 KB piece on "Drive B", then then next 128 KB piece on "Drive A" again, etc... In general you should leave it at the default of 128 K unless you are making a storage array which only deals with large files.

Rigs:
Defiant (Gamer)
Intrepid (HTPC)
Galaxy (Server)
07-27-2011 01:28 PM
Visit this user's website Find all posts by this user Quote this message in a reply
Steven S Offline
Forum Helper
*****

Posts: 1,069
Joined: Dec 2010
Reputation: 10
Thanks: 8
Given 84 thanks in 70 posts
 
Post: #6
RE: So you want to run a RAID array...
(07-27-2011 01:28 PM)Defiant Wrote:  
(07-27-2011 12:40 PM)Steven S Wrote:  the P67 RAID controller clearly shows a preference towards small file read speed. at first i thought the vastly different write vs read speeds on the SSD array is because of the SSDs themselves, but then, the mixed HDD array shows the same behavior, though not as pronounced.

I have a better explanation. RAID stripe size. What did you set it to? The smaller you set the RAID stripe size to the better it is for operating systems due to lots of small files, larger stripe sizes are great for when dealing with storage arrays that will store larger files (media or otherwise). Basically the stripe size determines how much data is written to each drive at a time, with a stripe of 128 KB a 1 GB file will have a 128 KB piece on "Drive A", then the next 128 KB piece on "Drive B", then then next 128 KB piece on "Drive A" again, etc... In general you should leave it at the default of 128 K unless you are making a storage array which only deals with large files.

the SSD array is at 32KB, while the HDD array is 128KB. i've tried pretty much every stripe size from 4KB to 128KB, and the same pattern persists for the SSD array. 32KB (and 16KB) happen to be the fastest for my SSDs, so i left it there. P67 RAID controller doesn't support stripe sizes greater than 128KB. otherwise, i would've tried larger sizes for the HDD array, since the ONLY thing stored there will be ridiculously large avi files.

(07-27-2011 01:28 PM)Peter M Dodge Wrote:  CC should carry some more professional grade RAID cards. I know I'd buy them, I have to go elsewhere ATM.

you can special order those cards if you wish, but given their specific usages and high price tags, they are definitely not something we stock in store.

Intel i7 2600K @ 4.5Ghz
16GB G.Skill Ripjaws Z 1600Mhz
Asrock P67 Fatal1ty Professional
MSI HD 7970 Lightning @ 1125/1600 MHz
Crucial M4 256GB SSD
3x Seagate Barracuda 1TB 7200.12 HDDs (RAID0)
Seagate Barracuda Green 2TB HDD
Silverstone Fortress FT02
(This post was last modified: 07-27-2011 01:49 PM by Steven S.)
07-27-2011 01:46 PM
Find all posts by this user Quote this message in a reply
Peter M Dodge Offline
Posts Dominator
*****

Posts: 5,010
Joined: Dec 2010
Reputation: 29
Thanks: 12
Given 138 thanks in 127 posts
     
Post: #7
RE: So you want to run a RAID array...
Special order would be fine, but CC doesn't have the ones I usually use in server builds available for special order as far as I can see.

[Image: currentbuild.png]
07-27-2011 02:05 PM
Visit this user's website Find all posts by this user Quote this message in a reply
David Offline
Posting Freak
*****

Posts: 1,197
Joined: Nov 2010
Reputation: 7
Thanks: 9
Given 58 thanks in 48 posts
 
Post: #8
RE: So you want to run a RAID array...
(07-27-2011 02:05 PM)Peter M Dodge Wrote:  Special order would be fine, but CC doesn't have the ones I usually use in server builds available for special order as far as I can see.

What are you looking for and I'll see what I can pull up.

The other Dave
Current Rig:

NZXT H440
MSI GTX 1080 X 8G
I7 6700K
MSI Z170A Gaming Pro Carbon
Corsair Vengeance LPX 2x16
Cooler Master Hyper 212 evo V2
Samsung Evo 128 SSD
Adata 240 SSD
WD 2TB Blue
07-27-2011 02:06 PM
Visit this user's website Find all posts by this user Quote this message in a reply
Peter M Dodge Offline
Posts Dominator
*****

Posts: 5,010
Joined: Dec 2010
Reputation: 29
Thanks: 12
Given 138 thanks in 127 posts
     
Post: #9
RE: So you want to run a RAID array...
(07-27-2011 02:06 PM)David Wrote:  What are you looking for and I'll see what I can pull up.

Well its nothing I'd need right now, but I'll go poke the ones I usually use in builds into the product suggest tool, if that actually gets looked at lol Tongue

[Image: currentbuild.png]
07-27-2011 02:07 PM
Visit this user's website Find all posts by this user Quote this message in a reply
Defiant Offline
Posting Leader
*****

Posts: 4,654
Joined: Nov 2010
Reputation: 64
Thanks: 59
Given 594 thanks in 458 posts
       
Post: #10
RE: So you want to run a RAID array...
(07-27-2011 02:07 PM)Peter M Dodge Wrote:  
(07-27-2011 02:06 PM)David Wrote:  What are you looking for and I'll see what I can pull up.

Well its nothing I'd need right now, but I'll go poke the ones I usually use in builds into the product suggest tool, if that actually gets looked at lol Tongue

Which ones would those be? I would be interested to hear what made you select them.

Rigs:
Defiant (Gamer)
Intrepid (HTPC)
Galaxy (Server)
07-27-2011 02:10 PM
Visit this user's website Find all posts by this user Quote this message in a reply
Post Reply