RAID Optimization Guide

[peaz]

IDE RAID seems to be the rage these days. Practically every motherboard manufacturer has several motherboards with IDE RAID features built-in. With RAID support built into most motherboards and hard disk prices dropping, more and more people are opting for RAID solutions as a cheap way to improve performance or reliability.

Today, we are going to take a look at how you can optimize your IDE RAID solution. Let's see what's the best RAID solution for your needs!

Link : RAID Optimization Guide!

[RAST]

Your write-up is very nice. I reviewed your findings and I have to say that there are three things where you might want to do a bit more research.

Just as some background on me, I’m a systems integration person and I’ve been in the industry since 1991. I’ve worked on SCSI attached RAID, SANs and all kinds of other stuff. My specialty is disk I/O tuning. Here are my three main critiques of the article.

1. RAID 0+1 does not have a “massive write penalty.” It as a 2 write penalty. You have to do 2 Write I/Os for each file system write. In other words when you save that word document each NTFS write I/O is doubled by the hardware. This makes sense because each side of the mirror has a full copy of the data. This sounds like a massive penalty until we get to the next topic.

2. RAID 5 is no panacea. It has some big drawbacks. First, you have a 4 write penalty (verses a RAID 0+1 two-write penalty). Why? The reason is complex but it has to do with failure states. I failed write to a RAID5 disk could leave the data corrupted. This is the worst possible outcome and must be avoided. Therefore each WRITE operation on a RAID5 disk requires 2 READ operations. 1 to read the existing data from the disk, 1 to read the existing parity from the disk. This is then stored in an internal buffer for later use. Next the RAID5 hardware must then Write the new data, then compute and write the new Parity. Finally the contents of the new write location is final and the controller can erase the old data it retrieved at the start of the write operation. (for more information on the RAID5 Write Penalty please see this link: http://searchstorage.techtarget.com/...286192,00.html) You can do a quick Google search on “RAID5 Write Penalty” to get a ton of articles on this.

3. Choosing a RAID level is not as easy as just going RAID5. There are times when RAID5 is more expensive than RAID0+1. These are in high random write environments like an Email server or SQL database. The 2 write penalty of RAID0+1 must be overcome by adding more disks. The 4 write penalty of RAID5 must be overcome by adding even MORE disks, twice as many!! There is a point where it is less expensive to add disks to the RAID0+1 because it requires fewer spindles to soak up the high write I/O. (for more information on choosing RAID based on I/O type see this link: http://is.pennnet.com/Articles/Artic...KEYWORD=schulz)

For your typical end user implementing an IDE RAID solution these points are not of the highest concerns. For a systems integrator dealing with thousands of write I/Os per second they are a critical performance issue.

[Techslacker]

Unless I missed it you failed to mention anything about block size. From what I've understood in the past on a single drive that has similar impacts on performance just as stripesize does with raid but now with raid you have to figure out the correct combination of the two to get ideal performance. Also I've heard through the grapevine something to the effect that the pagefile of Windows only gets read in 4Kb increments no matter the block size or anything. That too, if true, can play a limited role in performance tuning.

[peaz]

Hi guys. Thanks for your feedback. It's greatly appreciated.

RAST: I'll be looking into them and further research and update the article. Thanks again for pointing these points out.

Techslaker: Yes, block size was not tested as a variable in the tests of the article. Indeed i too think that block size may play a role in tuning the best performance out of a striped array. But I've decided to leave block size out of the scope and use the default NTFS block size instead due to the magnitude of the tests. However, expect a improved update on this guide in the near future.

In essence, I think that having the block size = optimal stripe size would be the best choice. that way, files fragmentation would be very minimal between the blocks and stripes.and if they have to be different, I'd figured that the block size should be round multiplications or divisions of stripe sizes. That means if you use the 16kb stripe size, the block size should be 4kb (4:1) , 8kb(2:1) or 16kb(1:1), 32kb(1:2). As for whether if more blocks per stripe is better or more stripes per block I can't say. I'd need to run tests to determine that.

Have anyone experimented on that?

[WildFrisco]

Originally Posted by Techslacker

Unless I missed it you failed to mention anything about block size. From what I've understood in the past on a single drive that has similar impacts on performance just as stripesize does with raid but now with raid you have to figure out the correct combination of the two to get ideal performance. Also I've heard through the grapevine something to the effect that the pagefile of Windows only gets read in 4Kb increments no matter the block size or anything. That too, if true, can play a limited role in performance tuning.

I think what you're referring to is what Windows refers to as allocation unit size. Everyone else refers to it as cluster size. By default when you format hard disks with 2K or XP, the default is usually 4KB. It's been suggested that this default cluster size is "useless" for raid 0. They also state anything besides 4KB is beneficial, like 16 and 32KB.

So how do you format and install 2K or XP with a custom cluster size (other than 4KB)? One way is to have an alternate system or boot drive. Boot with that drive, and access the disk manager in Windows. Select your new raid array and do something with it, LOL (can't remember the step; something like right click and set it as active or something). Then when you format it, you'll see Allocation Unit Size. Change it to something besides Default (4KB) and do a quick or regular format. Shut the computer down, unplug your alternate system or boot, and power up. Change the boot order to accordingly and begin the install. After you press F6 to load additional raid controller drivers and F8 to agree to Microsoft's EULA, when you get to the option of formatting the drive; select "Leave file system intact".

I think a little explanation on cluster or file allocation unit size on single drives and raid (0) arrays would be a useful update to this article. I could sure use it! I am in the process of tuning both my primary and secondary comps which are both on a raid 0 array and a gigabit network. For what? Don't ask .

[WildFrisco]

Originally Posted by peaz

However, expect a improved update on this guide in the near future.

How near?

[WildFrisco]

Originally Posted by peaz

In essence, I think that having the block size = optimal stripe size would be the best choice. that way, files fragmentation would be very minimal between the blocks and stripes.and if they have to be different, I'd figured that the block size should be round multiplications or divisions of stripe sizes. That means if you use the 16kb stripe size, the block size should be 4kb (4:1) , 8kb(2:1) or 16kb(1:1), 32kb(1:2). As for whether if more blocks per stripe is better or more stripes per block I can't say. I'd need to run tests to determine that.

Have anyone experimented on that?

This has been hotly debated in other forums (Can't recall which ones right now 'cause I'm about to pass out). It's a lot of work but I am interested in what may result with the Celeron system used in the article and maybe an ultra high end system (P4 socket 775 w/HT, two WD 74GB Raptors, two Seagate 7200.7 w/NCQ, or two Maxtor Diamond Max 10 w/NCQ [I think]). I'm too poor to own something like that.

[peaz]

Originally Posted by WildFrisco

This has been hotly debated in other forums (Can't recall which ones right now 'cause I'm about to pass out). It's a lot of work but I am interested in what may result with the Celeron system used in the article and maybe an ultra high end system (P4 socket 775 w/HT, two WD 74GB Raptors, two Seagate 7200.7 w/NCQ, or two Maxtor Diamond Max 10 w/NCQ [I think]). I'm too poor to own something like that.

The Celeron system have acutally been decommisioned. The article was written quite a while ago and is now being republished for the site.

Anyways. Once I've the equipments (2x raptor, 2x 7200 SATA, 2x 7200 pata, or more), then I can further investigate the impact of block sizes vs stripe sizes and also investigate RAID 5 performance impacts. Until then, well, i guess we all can still learn from each other via healthy discussions in this forum here

Personally, i would still think that one should decide which stripe size that is suitable for his/her needs. Then reformat that drive to the same stripe size for the block size (cluster size) But still, would need to benchmark 1st to really proof this point or 'unproof' it for the matter.

[Fiendishgames]

Complete RAID novice here.

I've inherited a PC with an Abit KT7-RAID motherboard (Highpoint RAID system) which I want to set up for number 1 son to use as a glorified jukebox.

I've got two identical 80 gig Maxtor drives for the RAID, and I was thinking of putting a third drive on the standard IDE slot for the purposes of storing disk images of the Maxtor drives.

Having read the excellent article on RAIDs I am a bit concerned that a RAID 0 set-up would see data on both drives lost if one of them fails. This would not be a problem if my Powerquest Drive Image (5.0) can do disk images of my drives (which I intend to partition) but will Drive Image be able to handle the striping?

Any advice gratefully accepted.


John

[SAMSAMHA]

nice guide for setting up raid, now if only if I can afford 2 identical HD to set them up. I got hd just not identical.