<?xml version="1.0" encoding="UTF-8"?><!-- generator="wordpress/1.5.1-alpha" -->
<rss version="2.0" 
	xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
	<title>Comments on: Death of RAID predicted, film at 11</title>
	<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/</link>
	<description>the blog that is not dansdata.com</description>
	<pubDate>Sun, 12 Feb 2012 09:25:33 +0000</pubDate>
	<generator>http://wordpress.org/?v=1.5.1-alpha</generator>

	<item>
		<title>by: RFC3251</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3400</link>
		<pubDate>Mon, 27 Oct 2008 04:40:12 +0000</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3400</guid>
					<description>&lt;code&gt;Losing one bit shouldn’t lose you more than one file. &lt;/code&gt;

Not quite. Losing one bit in one RAID stripe basically means you've lost all that stripe (due to the way RAID checksums work, the whole stripe will be marked as invalid). A typical stripe size is 64 kB, which must then be multiplied by the number of drives (minus one, for RAID-5). That will tell you how much data you've lost. If the total amount of data is smaller than one file system allocation unit, then you probably only lost one file (unless that part of the filesystem was compressed, wich can mean more than one file per allocation unit). If it was bigger, you may have lost more files.

Anyway, you are right that (unlike the ZDNet article claims) the risk is mainly to individual files (or small groups of files), not to the &lt;i&gt;whole&lt;/i&gt; array.</description>
		<content:encoded><![CDATA[	<p><code>Losing one bit shouldn’t lose you more than one file. </code></p>
	<p>Not quite. Losing one bit in one RAID stripe basically means you've lost all that stripe (due to the way RAID checksums work, the whole stripe will be marked as invalid). A typical stripe size is 64 kB, which must then be multiplied by the number of drives (minus one, for RAID-5). That will tell you how much data you've lost. If the total amount of data is smaller than one file system allocation unit, then you probably only lost one file (unless that part of the filesystem was compressed, wich can mean more than one file per allocation unit). If it was bigger, you may have lost more files.</p>
	<p>Anyway, you are right that (unlike the ZDNet article claims) the risk is mainly to individual files (or small groups of files), not to the <i>whole</i> array.
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Coderer</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3394</link>
		<pubDate>Sat, 25 Oct 2008 00:07:44 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3394</guid>
					<description>RE: Dan's comment about the coin not being fair -- you don't *really* have to worry until somebody calls &quot;edge&quot;.</description>
		<content:encoded><![CDATA[	<p>RE: Dan's comment about the coin not being fair -- you don't *really* have to worry until somebody calls "edge".
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Jonadab</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3393</link>
		<pubDate>Fri, 24 Oct 2008 23:43:09 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3393</guid>
					<description>Did I really write &quot;late 1998&quot;?  Make that &quot;late 2008&quot;.  And yeah, as of October 24th on Newegg, the cheapest 160GB drive runs about $41 (so, three of them would be $123 plus shipping), but for $58 you can get a 320GB drive (so two of them would be $116 plus shipping).  So unless you actually need the performance characteristics of RAID, which with a consumer-grade RAID controller are probably nothing to write home about anyway, the RAID1 setup is actually cheaper.

That does change when you get up into the larger capacities and higher performances of products that aren't really aimed at home users, so I'm not saying RAID5 doesn't have any real uses.  But for a typical home or small-office user, it's uneconomic.</description>
		<content:encoded><![CDATA[	<p>Did I really write "late 1998"?  Make that "late 2008".  And yeah, as of October 24th on Newegg, the cheapest 160GB drive runs about $41 (so, three of them would be $123 plus shipping), but for $58 you can get a 320GB drive (so two of them would be $116 plus shipping).  So unless you actually need the performance characteristics of RAID, which with a consumer-grade RAID controller are probably nothing to write home about anyway, the RAID1 setup is actually cheaper.</p>
	<p>That does change when you get up into the larger capacities and higher performances of products that aren't really aimed at home users, so I'm not saying RAID5 doesn't have any real uses.  But for a typical home or small-office user, it's uneconomic.
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Jonadab</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3392</link>
		<pubDate>Fri, 24 Oct 2008 23:30:42 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3392</guid>
					<description>Popup:  Just use a bignum library that stores numbers internally as rationals.

To me, the more interesting question is, &quot;Are we really going to lose the whole array if there's one unrecoverable section?&quot;  I don't know all the detailed mechanics of how RAID5 works, but that seems to me like it would require a remarkably boneheaded design.  Most ordinary single-disk filesystems don't have this property, certainly.  Even FAT filesystems usually have at least two copies of the allocation tables, or so I am told, and in practice a filesystem error is usually in a system file, which can be easily replaced because it's a standard part of the operating system and thus extremely non-unique.  I would like to think that in the event the RAID controller cannot recover a certain sector from the drives, it would just return an error when attempting to read that sector, the same as an ordinary disk controller would do if a single drive could not recover the sector, and let the software figure out what to do about it, which in most cases would probably just mean marking the sector as bad and possibly corrupting one file.  Perhaps someone who is more versed in RAID than I am can discuss this point.  (And before someone asks what if the whole RAID is just used to store One Big File, that would probably be a database, and in that case wouldn't you use replication instead of RAID?)

I do think that RAID5 can sometimes be a false economy, depending on the amount of storage capacity you're talking about.  Suppose you're talking about a three-drive RAID5 setup with a typical size of drive (in late 1998, we'll say 100GB, or maybe 150).  By using RAID5 instead of RAID1, you get 2/3 of the total capacity instead of 1/2, which sounds like a win.  But in a lot of cases it's probably a loss, because the three drives probably cost more than the two drives of greater capacity that you would need to get the same amount of storage out of RAID1.  The simplicity of RAID1 is worth something, as well.

It is, of course, still no substitute for offsite backups.  Any data you would still want in the event of a complete hardware failure of the entire computer should be backed up offsite.  Fire, theft, lightning, malicious bogus anonymous tips to overzealous law enforcement, ... there any number of exciting ways you can lose all the computers in the building at one shot.</description>
		<content:encoded><![CDATA[	<p>Popup:  Just use a bignum library that stores numbers internally as rationals.</p>
	<p>To me, the more interesting question is, "Are we really going to lose the whole array if there's one unrecoverable section?"  I don't know all the detailed mechanics of how RAID5 works, but that seems to me like it would require a remarkably boneheaded design.  Most ordinary single-disk filesystems don't have this property, certainly.  Even FAT filesystems usually have at least two copies of the allocation tables, or so I am told, and in practice a filesystem error is usually in a system file, which can be easily replaced because it's a standard part of the operating system and thus extremely non-unique.  I would like to think that in the event the RAID controller cannot recover a certain sector from the drives, it would just return an error when attempting to read that sector, the same as an ordinary disk controller would do if a single drive could not recover the sector, and let the software figure out what to do about it, which in most cases would probably just mean marking the sector as bad and possibly corrupting one file.  Perhaps someone who is more versed in RAID than I am can discuss this point.  (And before someone asks what if the whole RAID is just used to store One Big File, that would probably be a database, and in that case wouldn't you use replication instead of RAID?)</p>
	<p>I do think that RAID5 can sometimes be a false economy, depending on the amount of storage capacity you're talking about.  Suppose you're talking about a three-drive RAID5 setup with a typical size of drive (in late 1998, we'll say 100GB, or maybe 150).  By using RAID5 instead of RAID1, you get 2/3 of the total capacity instead of 1/2, which sounds like a win.  But in a lot of cases it's probably a loss, because the three drives probably cost more than the two drives of greater capacity that you would need to get the same amount of storage out of RAID1.  The simplicity of RAID1 is worth something, as well.</p>
	<p>It is, of course, still no substitute for offsite backups.  Any data you would still want in the event of a complete hardware failure of the entire computer should be backed up offsite.  Fire, theft, lightning, malicious bogus anonymous tips to overzealous law enforcement, ... there any number of exciting ways you can lose all the computers in the building at one shot.
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Major Malfunction</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3388</link>
		<pubDate>Fri, 24 Oct 2008 19:28:20 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3388</guid>
					<description>&quot;As with Major Malfunction’s comment, in the real world if you flip a coin 91 times and it comes up heads every time, you’re actually likely to suspect there’s something fishy going on, making the coin not entirely fair :-).&quot;

Au contraire! The coin was fair!

It's the *technique* that was not. *That's* about as random as juggling chainsaws.</description>
		<content:encoded><![CDATA[	<p>"As with Major Malfunction’s comment, in the real world if you flip a coin 91 times and it comes up heads every time, you’re actually likely to suspect there’s something fishy going on, making the coin not entirely fair :-)."</p>
	<p>Au contraire! The coin was fair!</p>
	<p>It's the *technique* that was not. *That's* about as random as juggling chainsaws.
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Mohonri</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3385</link>
		<pubDate>Fri, 24 Oct 2008 11:10:28 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3385</guid>
					<description>Well, rats.  Peridot beat me to my point about e.  When you mentioned that a 1/1,000,000 chance 1,000,000 times was 36.8%, I immediately recognized that magic number (e^-1).

It takes around 5 million tries to get the probability of at least one failure up to 95%.</description>
		<content:encoded><![CDATA[	<p>Well, rats.  Peridot beat me to my point about e.  When you mentioned that a 1/1,000,000 chance 1,000,000 times was 36.8%, I immediately recognized that magic number (e^-1).</p>
	<p>It takes around 5 million tries to get the probability of at least one failure up to 95%.
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Griffyn</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3384</link>
		<pubDate>Fri, 24 Oct 2008 08:23:46 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3384</guid>
					<description>The article also said something about RAID 6 not helping much (RAID 6 has two lots of parity distributed across the disks, instead of just 1 for RAID 5), as it just delays the inevitable.  But surely - even when the figures are made correct - this means that for RAID 6 to not help much, that it would require two read errors to occur on two disks in the same stripe.  I think the article writer missed that point and simply merged all the numbers together.</description>
		<content:encoded><![CDATA[	<p>The article also said something about RAID 6 not helping much (RAID 6 has two lots of parity distributed across the disks, instead of just 1 for RAID 5), as it just delays the inevitable.  But surely - even when the figures are made correct - this means that for RAID 6 to not help much, that it would require two read errors to occur on two disks in the same stripe.  I think the article writer missed that point and simply merged all the numbers together.
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Jax184</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3383</link>
		<pubDate>Fri, 24 Oct 2008 04:58:32 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3383</guid>
					<description>Now hang on a moment...

The article claims this failure rate will lead to the DEATH of RAID, but wouldn't it be more likely to cause the LIFE of RAID? If a drive has a decent chance of corrupting 1 file every time you fill it up, wouldn't you want to put it into a RAID 5 even more than ever before?</description>
		<content:encoded><![CDATA[	<p>Now hang on a moment...</p>
	<p>The article claims this failure rate will lead to the DEATH of RAID, but wouldn't it be more likely to cause the LIFE of RAID? If a drive has a decent chance of corrupting 1 file every time you fill it up, wouldn't you want to put it into a RAID 5 even more than ever before?
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Daniel Rutter</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3381</link>
		<pubDate>Fri, 24 Oct 2008 03:53:51 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3381</guid>
					<description>I don't think drive failure and unrecoverable read errors are the same. I mean, yes, any unrecoverable error is a drive failure, but a drive that's failed to read something is not necessarily a dead drive.

As far as I know, unrecoverable read errors always happen &lt;b&gt;after&lt;/b&gt; retry attempts. Sometimes the drive just can't figure out what that bit used to be. I think it's quite possible for the drive to then be able to successfully write and read data from that exact same location, though I presume it'll actually map that location out, probably replacing it with a sector from a section set aside for the purpose.</description>
		<content:encoded><![CDATA[	<p>I don't think drive failure and unrecoverable read errors are the same. I mean, yes, any unrecoverable error is a drive failure, but a drive that's failed to read something is not necessarily a dead drive.</p>
	<p>As far as I know, unrecoverable read errors always happen <b>after</b> retry attempts. Sometimes the drive just can't figure out what that bit used to be. I think it's quite possible for the drive to then be able to successfully write and read data from that exact same location, though I presume it'll actually map that location out, probably replacing it with a sector from a section set aside for the purpose.
</p>
]]></content:encoded>
				</item>
	<item>
		<title>by: Alereon</title>
		<link>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3380</link>
		<pubDate>Fri, 24 Oct 2008 03:37:17 +0100</pubDate>
		<guid>http://dansdata.blogsome.com/2008/10/23/death-of-raid-predicted-film-at-11/#comment-3380</guid>
					<description>One factor I'm not sure is being considered is that drive failure rates (which is what unrecoverable errors are, if the drive didn't fail the read could be retried successfully) aren't correlated with volume of data read, which makes 1-error-in-12TB statistics meaningless. Drive failure rates are also highly variable over the life of the drive, starting off at their maximum during the break-in period, then dropping to their normal level, and then steadily increasing as the drive exits its useful service life. I find it easy to believe that a RAID5 array built with new drives fresh out of the box could experience multiple drive failures during a short period of time, but after six months this becomes substantially less likely. This is also why you keep hot-spares, so the drive you are swapping in isn't still in its break-in period.</description>
		<content:encoded><![CDATA[	<p>One factor I'm not sure is being considered is that drive failure rates (which is what unrecoverable errors are, if the drive didn't fail the read could be retried successfully) aren't correlated with volume of data read, which makes 1-error-in-12TB statistics meaningless. Drive failure rates are also highly variable over the life of the drive, starting off at their maximum during the break-in period, then dropping to their normal level, and then steadily increasing as the drive exits its useful service life. I find it easy to believe that a RAID5 array built with new drives fresh out of the box could experience multiple drive failures during a short period of time, but after six months this becomes substantially less likely. This is also why you keep hot-spares, so the drive you are swapping in isn't still in its break-in period.
</p>
]]></content:encoded>
				</item>
</channel>
</rss>

