How To Spot A Psychopath

This little guy’s hilarious.

(Yet another Crabfu creation, of course.)

From a reader:

Love to hear your take on this: Why RAID 5 stops working in 2009.

That article by Robin Harris is more than a year old, but was suddenly linked hither and yon in the last few days. Its thesis is that because RAID array capacities are approaching unrecoverable read error rates, if one disk in your RAID fails, you’ll very probably get a read error from one of the other disks in the course of rebuilding the array, and lose data.

This basic claim is true, but there are three reasons why this problem is not as scary as it sounds.

1: Losing one bit shouldn’t lose you more than one file. Consumer RAID controllers may have a fit over small errors, but losing one file because a drive failed is seldom actually a serious problem.

2: Putting your valuable data on a RAID array and not backing it up is a bad idea even if disk errors never happen. One day, you are guaranteed to confidently instruct your computer to delete something that you actually want, and RAID won’t protect you from that, or from housefires, theft of the drives, and so on. You still need proper backups.

3: If you’re going to build a story on statistics, it helps a lot if you get the statistics right.

Robin Harris says it is “almost certain” that 12 terabytes of disk drives with a one-in-12Tb read error rate will have an error if you read the whole capacity.

This statement is wrong.

Actually, the probability of one or more errors, in this situation, is only 63.2%. When you know why this is, you discover that there’s less to fear here than you’d think.

(Robin Harris is not the only sinner, here. This guy makes exactly the same mistake. This guy, on the other hand, says one error in ten to the fourteen reads gives you “a 56% chance” of an error in seven terabytes read; he’s actually done the maths correctly.)

The mistake people make over and over again when figuring out stuff like this is saying that that if you’ve got (say) a one in a million chance of event Y occurring every time you take action X, and you do X a million times, the probability that Y will have happened is 1.

It isn’t.

(If it were, then if you did X a million and one times, the probability that Y will have occured would now be slightly more than one. This is unacceptably weird, even by mathematicians’ standards.)

What you do to figure out the real probabilities in this sort of situation is look at the probability that Y will never happen over your million trials.

(If it matters to you if Y happens more than once, then things get more complex. But usually the outcomes you’re interested in are “Y does not happen at all” and “Y happens one or more times”. That is the case here, and in many other “chance of failure” sorts of situations.)

To make this easier to understand, let’s look at a version of the problem using numbers that you can figure out on a piece of paper, without having to do anything a million times.

Let’s say that you’re throwing an ordinary (fair!) six-sided die, and you don’t want to get a one. The chance of getting a one is, of course, one in six, and let’s say you’re throwing the die six times.

For each throw, the probability of something other than one coming up is five in six. So the probability of something other than one coming up for all six throws is:

5/6 times 5/6 times 5/6 times 5/6 times 5/6 times 5/6.

This can more easily be written as five-sixths to the power of six, or (5/6)^6, and it’s equal to (5^6)/(6^6), or 15625/46656. That’s about 0.335, where 1 is certainty, and 0 is impossibility.

So six trials, in each of which an undesirable outcome has a one in six chance of happening, certainly do not make the undesirable outcome certain. You actually have about a one-third chance that the undesirable outcome will not happen at all.

It’s easy to adjust this for different probabilities and different numbers of trials. If you intend to throw the dice 15 times instead of six, you calculate (5/6)^15, which gives you about a 0.065 chance that you’ll get away with no ones. And if you decide to toss a coin ten times, and want to know how likely it is that it’ll never come up tails, then the calculation will be (1/2)^10, a miserable 0.00098.

In the one-in-a-million, one-million-times version, you figure out (1 - 1/1000000)^1000000, which is about 0.368. So there’s a 36.8% chance that the one-in-a-million event will never happen in one million trials, and a 63.2% chance that the event will happen one or more times.

OK, on to the disk-drive example.

Let’s say that the chance of an unrecoverable read failure is indeed one in ten to the 14 - 1/100,000,000,000,000. I’ll express this big number, and the other big numbers to come, in the conventional computer-y form of scientific notation that doesn’t require little superscript numbers. One times ten to the power of 14, a one with 14 zeroes after it, is thus written “1E+14″.

The chance of no error occurring on any given read, given this error probability, is 1 - 1/(1E+14), which is 0.99999999999999. Very close to one, but not quite there.

(Note that if you start figuring this stuff out for yourself in a spreadsheet or something, really long numbers may cause you to start hitting precision problems, where the computer runs out of digits to express a number like 0.99999999999999999999999999999999999999 correctly, rounds it off to one, and breaks your calculation. Fortunately, the mere fourteen-digit numbers we’re working with here are well within normal computer precision.)

OK, now let’s say we’re reading the whole of a drive which just happens to have a capacity of exactly 1E+14 bits, at this error rate of one error in every 10^14 reads. So the chance of zero errors is:

(1 - 1/(1E+14))^1E+14

This equals about 0.368. Or, if you prefer, a 63.2% chance of one or more errors.

Note that the basic statement about the probability of an error remains true - overall, a drive with an Unrecoverable Read Error Rate of one in ten to the fourteen will indeed have such an error once in every ten to the fourteen reads. But that doesn’t guarantee such an error in any particular ten to the fourteen reads, any more than the fact that a coin comes up evenly heads or tails guarantees that you’ll get one of each if you throw it twice.

Now, a RAID that’s 63.2% likely to have an error if one of its drives fails is still not a good thing. But there’s a big difference between 63.2% and “almost certain”.

(Note also that we’re talking about a lot of data, here. At fifty megabytes per second, ten to the fourteen bits will take about 2.8 days to read.)

Getting the statistics right makes the numbers look proportionally better if the error rate can be reduced.

If drive manufacturers manage to reduce the error rate by a factor of ten, for instance, so now it’s one in every ten to the fifteen reads instead of every 1E+14, the chance that you’ll get no such errors in a given ten to the fourteen reads improves to about 90.5%.

If they reduce the error rate all the way to one in ten to the sixteen, then ten to the fourteen reads are 98.9% likely to all be fine.

I’m not saying it’s necessarily easy to make such an improvement in the read error rate, especially in the marketing-bulldust-soaked hard-drive industry.

But neither is the situation as dire as the “almost certain” article says.

All who commit such crimes against mathematical literacy are hereby sentenced to read John Allen Paulos’ classic Innumeracy.

(This is not a very severe sentence, since the book is actually rather entertaining.)

On the subject of objects that look like alien technology, I’ve got a Piet Hein “Super Egg” drink cooler, too.

I got it at a decent discount when ThinkGeek were clearing their stock; they don’t have them any more, but the cooler and umpteen other “superellipse“-shaped products have been on sale from various overpriced homewares places for decades.

The superellipse is like a hybrid between synthetic-rectangular and natural-circular, as explained in this Scientific American article, which was written by the inimitable Martin Gardner more than forty years ago (I just re-read Fads and Fallacies the other day).

And Mr Hein had a real bee in his bonnet about superellipses. He designed superellipse-shaped salt-shakers, bowl sets, candlesticks, plates… you name it.

(Sorry about the stupid window-within-a-window thing in the piethein.com links, by the way; that’s just the way that site works.)

Despite all the folderol in the Super Egg drink cooler’s rather tongue-in-cheek instruction sheet, as far as I can see it does not actually seem to be very good at cooling drinks. The enthalpy of fusion of water ice is hard to beat; a little stainless-steel egg with a mysterious liquid inside just can’t achieve much, unless you chill it so far that it’ll crust itself up with ice after you put it in your glass.

But it’s nonetheless a neat little object, being both geometrically interesting and mysterious-sounding, on account of the liquid that sloshes around inside when you shake it. And it does indeed neither dilute your drink, nor change its flavour in any other way.

(Many sites say the liquid inside the cooler is meant to freeze, but I don’t think that’s likely to happen at home-freezer temperatures. Perhaps that’s what you have to do to get the cooler to work properly.)

Uri Geller was, apparently, given a gold Piet Hein cooler by John Lennon, who (Uri says) spun a brilliant tale about how the object was given to him by bug-faced aliens.

I suppose it’s possible that Lennon had a weird hallucination (in this case, possibly even without chemical assistance…), then found the drink cooler lying around.

I prefer, however, to think that Lennon knew exactly what the mysterious object was, and was just taking the piss out of Uri.

Now that Lego has two kinds of track piece, it’s easier to make similar tracked vehicles in very different scales.

Via TechnicBricks, a frequent link-target for me recently, behold the last unknown Technic Lego set for 2009:

It’s the one in the top-left corner, being dwarfed by the 8256 Super Kart and 8262 Quad-Bike. It’s called the 8259 Mini Dozer, and it’s…

…completely anerable.

Compare and contrast with its (much) bigger brother…

…the 8275 Motorized Bulldozer. The Mini Dozer is, basically, a micro-scale version of 8275.

So now you can put 8259 on one end, 8275 on the other, and the classic 856 in the middle, and have flying-ducks bulldozers!

(I wonder how long it’ll be before someone finds room for a Micro Motor inside an 8259?)

UPDATE: I know what you’re thinking. You’re thinking, “I won’t believe this kit even exists unless I can fly around it using Microsoft Photosynth and see every tiny detail.”

No problem.

UPDATE 2: It didn’t take long for people to start building the set before you can buy one.

It is, as mentioned in this post, not entirely easy to get Lego tracks - either the new big or old small ones, or ones improvised from other pieces - to grip the ground well. They’re particularly lousy on smooth surfaces, like indoor floors or table-tops.

I am indebted to the excellent TechnicBricks blog once again for showing me that it’s actually easy to increase the traction of the new-style treads. Just loop rubber bands around each link, or every second link if you want to avoid bands rubbing on each other.

OK, now I’ve definitely got to buy a bucketload of links, and make something that can climb stairs.

From: “Kyra jhons” <kyra.jhons@beacaliforniaspermdonor.com>
To: Blogsome <dan@dansdata.com>
Subject: Page rank 4 link request
Date: Wed, 15 Oct 2008 15:40:41 -0500

Hi:

My name is Kyra Jhons, I´ve visited your website blogsome.com and I
was wondering if you would like to exchange links with my website,
currently I have a Business website and I´m looking for other similars
like yours. In exchange I’ll give you a link from my
“Beacalifornias Permdonor Marketing Service” website with page rank 4
(http://www.beacaliforniaspermdonor.com).

Your link will be exactly here:

http://www.beacaliforniaspermdonor.com with page rank 4 (your link will
be in Homepage and NOT at links page!!)

If you are interested please add my link to your site using the
following details, let me know once it’s ready and dont forget to send
me your site details for do the same for you, your link will be ready
on my site in less than 24 hours, otherwise you can delete my link from
your site.

Title: UK Prepaid Cards
Url: http://www.what-prepaid-card.co.uk
Description: What Prepaid compares current UK pay as you go prepaid
credit cards.

Or you can use the following html code:

<a href=”http://www.what-prepaid-card.co.uk”>UK Prepaid Cards</a> -
What Prepaid compares current UK pay as you go prepaid credit cards.

Please let me know once it’s ready and send me your site details for do
the same for you. I’ll be waiting for your kind reply.

Best Regards

Kyra Jhons

PD: In order to follow anti-spam regulations, please be so kind of
filling in the following form if you don’t want to receive any more
messages from this address.
http://www.goodeyeforlinks.com/Contact_Us.html

The link-farm site and the site they want promoted are different because these spammers, like many others, are playing the triangle. I think beacaliforniaspermdonor.com may set a new high-water mark for link irrelevance, pretty much regardless of who gets this spam.

Oh, and in case you’re wondering, I am not the boss of Blogsome. And I’m certainly not going to hand out links to any Johnny-come-lately who doesn’t even offer me “f-ree software“.

I particularly like how they call their link-farm page “Beacalifornias Permdonor”. It’s like the exact opposite of the Who Represents/Experts Exchange thing. (See also “beontopranking-google.com“.)

Australians will be unable to opt-out of the government’s pending Internet content filtering scheme, and will instead be placed on a watered-down blacklist, experts say.

Under the government’s $125.8 million Plan for Cyber-Safety, users can switch between two blacklists which block content inappropriate for children, and a separate list which blocks illegal material.

If this actually happens, then it’ll be a considerable pain. I’d just switch to using Tor or something to avoid being unable to see “illegal” content (it should be for the courts to decide if a page is “illegal”, of course…), but that’s pretty bleeding slow.

(On account, of course, of the vast numbers of office workers downloading BlackBackdoorBimbosAndTheirBarnyardBeaus27.avi.)

But I’m pretty sure it’s not going to happen.

Similar threats have been made here in the past, and they’ve always petered out into nothing. There are no votes to be won in actually filtering the Internet, after all. The people who vote based on Net filtering promises are unable to tell whether it’s actually happening or not. And there are plenty of votes to be lost when everyone who doesn’t call their browser “the Internet” discovers that they can’t get to YouPorn or Mininova any more.

There’s not even much money to be made in making filtering software that actually works. The big bucks in content filtering were and are based on arse-covering and plausible deniability, not actually stopping anybody from seeing anything in particular.

I confidently predict that this will just end up being another easily-circumvented waste of taxpayers’ money.

There are rats under our floor.

I don’t care about that, per se. I think it’s cute when I put the mouldy end of a bread-loaf in the compost bin and the next day the inside of it’s all been eaten out into a cosy little cave-of-food.

(Anne does not think this is cute in any way at all.)

I think rats are cute too, even ordinary brown ones that want to bite you.

(Anne believes I may need to adjust my medication.)

Unfortunately, though, the rats keep weeing conductively on important parts of the heating system, and chewing other important parts of it.

There are four cats in this house.

So a solution suggests itself.

But that would (a) mean, at best, slow death by torture for little fuzzy creatures which I do not want to eat and (b) expose our own precious furry child-substitutes not only to the risk of loss of self-esteem, should they find themselves unable to catch even the doziest rats, but also to the dangers of the outdoors. Never mind being hit by cars; for all we know, there’s some toxic something-or-other growing somewhere around the house that killed poor Mickey.

So from now on, no cat of ours goes outdoors unless escorted by at least six Secret Service agents.

But the rats have got to go.

So I purchased no-kill traps (from this guy; the traps are cheap and work fine, but they come flat-packed and must be cable-tied into shape). A bit of peanut butter on bread for bait, and bang, one rat was caught in almost no time.

Into the car (on some newspaper…) the trapped rat went, and he or she and I enjoyed a brief but stimulating drive to Kingsford Smith Memorial Park, from the verges of which the rat has by now almost certainly darted into somebody else’s house.

When I got back, re-baited the trap and set it up again, I found a rat in the other trap.

Lather, rinse, repeat.

When I took that second trap back down, the first one was still empty. So I presume we have now caught the two stupidest rats, and will never catch another.

Perhaps we should get a goanna, or something.

I know you were all perched on the edge of your seats about that Moletech, or possibly MTECH, Fuel Saver thing.

Well, The Western Australian Department of Consumer and Employment Protection, or NAMBLA DOCEP, has reached an “undertaking” with the two companies responsible for The MoleWhatever Fuel Saver, in which those companies agree to stop selling their useless gadget in Western Australia and DOCEP agree to not kick their miserable scamming arses into the Indian Ocean.

(I paraphrase lightly. Here’s the DOCEP page about this. I’ve also got a copy of the official PDF press release here.)

I don’t know whether the Federal Government has reached an opinion about Moletech, but it didn’t look good for them in January.

The Western Australian developments were brought to my attention by the proprietor of the Thinkingisreal blog, who saw a story about the “undertaking” on the WA edition of of the sterling tabloid-TV current affairs program Today Tonight.

Today Tonight and their cousins at A Current Affair appear to decide whether to run an approving or a scathing story about nonsense diets, umpteen useless fuel savers, and psychics, by flipping a coin. Actually, I think dice may be involved, with a roll of 24 or higher needed to get a critical story.

But TT are really solidly committed to this story. Just look at their Consumer Protection page!

In case you’re coming to this post a while after I wrote it and that page now actually has some content, be advised that at the time of writing, and since TT ran the story, the sum total of the non-navigation content on that page - which is presumably meant to provide background information for every consumer-protection story the show has ever run - is:

Fuel Saver Ban
Consumer Protection
1300 30 40 54

Seriously, that’s it.

They don’t even say what Fuel Saver they’re talking about.

Awesome work, guys. Bonuses all round.

(I searched for other pages on the 7perth.com.au site that mentioned this, and found the same “Fuel Saver Ban” snippet on this page, which contains what looks like a nose-to-tail site-content dump. The title of that page is - again, if the page isn’t there by the time you read this, be advised that I am not making this up - “Alzheimers Cure”. And the page-content below the “Fuel Saver Ban” snippet is about a spray-on cure for arthritis pain that uses “Herbal Synergism”. Two pages-worth up from the Fuel Saver snippet is… a miracle diet, this time based around milk protein. Magnificent.)

Thinkingisreal had a blog post up about this, but pulled it because there wasn’t yet any solid info about the ban on the Today Tonight or DOCEP sites (the press release was mentioned on this DOCEP page, but the link to it was broken. Now the official statement is up. Here’s DOCEPs list of current media statements).

Anyway, apparently Today Tonight did a previous story on the Moletech gadget, in which they found “promising results” in their entirely science-free investigation. That story is still proudly mentioned on the home page of moletech.us.

(I originally thought TT had, being at least slightly honest, mentioned this previous story in the most recent one. Thinkingisreal says he doesn’t actually remember them doing so.)

But now, wouldn’t you know it, TT have changed their minds, and decided that this zillionth example of a fuel conditioner that’s supposed to work by some sort of molecular balderdash (”nano negative ions!”) is just as useless as all the rest.

That quote from Band of Brothers springs to mind, yet again.

Picture credit: Conlawprof

A reader asks:

Simple, maybe stupid question. We had an Olympus OM-10 which broke down, and some good lenses and stuff which didn’t. Please, do OM-10 lenses fit on modern Olympus digital cameras? I asked Olympus but they didn’t answer.

Patrick

In brief: Yes, they do. Just buy an OM-System-to-Four-Thirds adapter ring and away you go. Olympus make their own adapter, and there are cheap Chinese ones that’re probably just as good, since there’s no glass in there.

EBay’s full of adapters for popular camera and lens types, but there are far fewer Four Thirds cameras out there than Canons or Nikons, so there only seem to be a couple of OM-to-Four-Thirds adapters on eBay at the moment. I wouldn’t be worried at all about buying one on eBay for $40 delivered rather than from Olympus for $100, though; when there’s no glass in the adapter, it’s hard to get it wrong.

Olympus have a list of recommended lenses for use with the adapter. Other lenses should also work, but may lose a little quality.

The reason for this is that film responds in pretty much exactly the same way regardless of the angle from which light hits it - well, as far as SLR-camera applications go, anyway. Digital sensors, on the other hand, have their own array of tiny “microlenses” over the actual sensor pixels, not to mention protective glass and anti-aliasing filters on top of the microlenses. This stuff does not respond the same to light coming at an angle as it does to light coming straight at it - think of looking at an LCD monitor from an angle, versus looking at a CRT.

So for best results on digital cameras, you need lenses that’re as close as possible to being “telecentric“, which means the light coming out of the back of the lens is lined up, as much as possible, with the axis of the optical components.

All of the standard Four Thirds lenses are pretty telecentric. OM System lenses aren’t so much, because they didn’t need to be to work fine with film.

Olympus have a page about this issue, too.

(Telecentricity may be less of a problem as digital sensors evolve - see this page for some speculation.)

Worrying about telecentricity is a bit nit-picky, particularly because the part of the image circle where the rays from any lens are likely to be least perpendicular to the sensor are around the edge, which consumer digital cameras with their “APS“-sized sensors can’t even see.

The down side of this is that an APS-sensor camera can only see about two-thirds of the image a “full frame” sensor or 35mm film frame would capture with the same lens. This means all of your lenses appear more telephoto, and you need a serious bug-eyed-monster lens to get really wide-angle photos. The up side is that, even if you don’t care about telecentricity, most lenses store their problems around the outside of the image circle. Vignetting, chromatic aberration, softness; all are worst around the edges, which simply aren’t seen at all by an APS-sensor DSLR using 35mm-film lenses.

The Olympus/Kodak Four Thirds system was actually purpose-built around these APS-sized sensors, which is why its lenses and cameras are smaller and lighter than those for other big-brand DSLRs.

(The OM System cameras and lenses were small and light compared with the competition too, though that was because of ingenious engineering rather than just having smaller film.)

“Mainstream” Canon and Nikon (and Sigma, for that matter) DSLRs can have full-35mm-frame-sized sensors, but the affordable models only actually have the smaller APS size. So, for those cameras, the lenses and bodies are bigger than they need to be. You can get lenses that only work (well, only work properly, at least) with APS sensors (Canon’s EF-S line, for instance). But most mainstream DSLR lenses, and all of the really high-quality Canon and Nikon ones, still throw a 35mm-sized circle of light into the camera, even if only the APS-rectangle middle of it is being caught by the sensor.

The critical issue for putting a lens made for one type of camera on another one - no matter what company makes the camera and the lens - is how far from the back of the lens the film, or sensor, is expected to be. This is called the “lens register” or “registration distance”. The adapter you use to attach System X Lens to System Y Body will inescapably add a bit of distance of its own, so you need System Y’s registration distance to equal System X’s distance plus the thickness of the adapter.

If System Y’s register is not big enough - if it’s smaller than System X’s, or so close to it that even a skinny adapter will move the lens too far away from the sensor - then it’s still possible to adapt the foreign lens onto the camera, but only with some serious limitations.

If the lens is just too far away from the sensor then you won’t be able to focus to infinity - but you will be able to focus closer than you’d otherwise be able to. This is what people do on purpose when they add “extension tubes” or bellows to lenses for macro work.

(Note that this may slightly hurt image quality, since aberration-correction expects the sensor or film to be the normal distance behind the lens. A lens that perfectly focusses red, green and blue right on top of each other at the normal registration distance probably won’t do that any more if you move it further away from the sensor. The image-quality loss should be much smaller than what you’d get from a simple screw-on front-of-lens “magnifying glass” macro adapter, though.)

If you want to keep infinity focus with a lens that’s too far from the sensor, you’ll need not a simple ring adapter, but an adapter with optics in it to increase the lens’s registration distance. Unless that adapter is rather expensive, this will hurt image quality so much that you’d be better off getting a cheap and nasty lens with similar specifications that was made to fit your camera in the first place.

Karen Nakamura’s Photoethnography.com has an excellent page about inter-system lens compatibility, with register numbers for many camera types.

Let’s pretend you’ve bought a Canon EOS (EF) digital SLR, and want to put your Olympus lenses on it.

The registration for EOS cameras is 44 millimetres, and for the OM System is 46mm, so it’s possible to put the latter lens on the former camera - but only if your lens adapter is a mere 2mm in thickness, or contains the dreaded optics.

It turns out that it is indeed possible to make adapter rings that’re this thin; you can buy ‘em quite cheaply on eBay. You can even get “AF Confirm” versions, which allow the camera’s autofocus system to beep when it reckons you’ve got the scene in focus, just as it does if you’re using a Canon lens in manual-focus mode. The focus screens in mass-market DSLRs are usually not very helpful for manual focussing, and their viewfinders also commonly aren’t very big and bright, so AF confirm can be more useful than you might think.

(Photo by Andy Crowe, coincidentally taken through the viewfinder of a Four Thirds DSLR with an OM lens. This is an upgraded focus screen, not the one that came with the camera.)

The situation for OM lenses on Four Thirds cameras is rather easier. OM System 46mm, Four Thirds about 38.7mm; the adapter has to be about 7.3mm in thickness, which makes it quite easy to make.

Picture credit: David Reeves

Simple adapters are inadequate if the lens you’re adapting needs to be controlled by the camera to work properly. Modern autofocus lenses, for instance, may still have a manual focus mode (possibly without even a distance scale), but they usually don’t have a control on the lens to set aperture. To set the aperture to anything other than wide open, therefore, you need to attach the lens to a camera of its native type, select the aperture you want, press the depth-of-field-preview button to make the lens stop down, then remove the lens while still holding the button, so it stays at that setting. This is not very practical.

If you’re just putting old all-manual lenses on new cameras, though, these sorts of problems don’t arise. The lens controls are all on the lens, so it’ll work as it did before.