Comments on: The Copy Of Doom http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/ the blog that is not dansdata.com Sun, 12 Feb 2012 06:34:10 +0000 http://wordpress.org/?v=1.5.1-alpha by: frasera http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4301 Sat, 14 Feb 2009 11:01:32 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4301 hm nice teracopy recommendations:) helped me a bit with network transfers of large files. hm nice teracopy recommendations:) helped me a bit with network transfers of large files.

]]> by: Jonadab http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4234 Sun, 08 Feb 2009 12:03:18 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4234 Obviously, no software yet invented can supply the performance you *really* want in this situation, but yeah, consumer versions of Windows are particularly lame about it. A better approach (on the part of the OS) is to copy large files in large-chunk-at-once mode but give your-chunk-is-next scheduling priority to small chunks, which will then be smaller files. That would probably have greatly reduced the collateral damage in your situation, though the large file copies would still have taken a goodly while to complete. Various OSes have various perf characteristics, but I suspect almost any server-grade OS will do better here than Windows XP. Ideally you don't want to use Windows desktops as file servers, but you know, sometimes you've GOT the computer sitting around turned on almost all of the time anyway, and it happens to have a bunch of free space on its drive, and it's just so tempting to put a shared folder on there and use some of that available space. I do that myself. The protocol does also matter, but if you want to access the files from desktop Windows systems, you pretty much have only one choice there (CIFS). I believe Microsoft supports other protocols (e.g., NFS) in some of its server-OS products, but if Windows XP or Vista can connect to NFS shares, it's news to me. Obviously, no software yet invented can supply the performance you *really* want in this situation, but yeah, consumer versions of Windows are particularly lame about it.

A better approach (on the part of the OS) is to copy large files in large-chunk-at-once mode but give your-chunk-is-next scheduling priority to small chunks, which will then be smaller files. That would probably have greatly reduced the collateral damage in your situation, though the large file copies would still have taken a goodly while to complete.

Various OSes have various perf characteristics, but I suspect almost any server-grade OS will do better here than Windows XP. Ideally you don't want to use Windows desktops as file servers, but you know, sometimes you've GOT the computer sitting around turned on almost all of the time anyway, and it happens to have a bunch of free space on its drive, and it's just so tempting to put a shared folder on there and use some of that available space. I do that myself.

The protocol does also matter, but if you want to access the files from desktop Windows systems, you pretty much have only one choice there (CIFS). I believe Microsoft supports other protocols (e.g., NFS) in some of its server-OS products, but if Windows XP or Vista can connect to NFS shares, it's news to me.

]]> by: Daniel Rutter http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4223 Fri, 06 Feb 2009 12:06:40 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4223 Yeah, one more time: It wasn't me doing the copying. I wouldn't have started umpteen parallel copies, if only because of the obvious horrible inefficiency. Yeah, one more time: It wasn't me doing the copying.

I wouldn't have started umpteen parallel copies, if only because of the obvious horrible inefficiency.

]]> by: Coderer http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4222 Fri, 06 Feb 2009 04:50:46 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4222 Dan: you of all people should know the CTRL-click-everything-you-want-before-copying rule. Windows makes a nice, sane, orderly queue for copies *if* you do it all in one go. Dan: you of all people should know the CTRL-click-everything-you-want-before-copying rule. Windows makes a nice, sane, orderly queue for copies *if* you do it all in one go.

]]> by: Alex Whiteside http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4216 Wed, 04 Feb 2009 22:29:08 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4216 Times like this, Windows needs someone like Mr. Resetti to pop up and berate you for corrupting the file table, then encourage you to restore from the backups you'd better have made, because otherwise he's coming after you with a rake. Not for Dan's benefit of course but it would be a more effective deterrent than an enforced chkdsk. It's shocking in this day and age that OSes don't have a built in sense for which disk jobs simply should not be done in parallel. Times like this, Windows needs someone like Mr. Resetti to pop up and berate you for corrupting the file table, then encourage you to restore from the backups you'd better have made, because otherwise he's coming after you with a rake. Not for Dan's benefit of course but it would be a more effective deterrent than an enforced chkdsk.

It's shocking in this day and age that OSes don't have a built in sense for which disk jobs simply should not be done in parallel.

]]> by: allsub http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4214 Tue, 03 Feb 2009 09:35:53 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4214 Ah, the kind of problem I ran into back in the good old days of network sharing mp3's in my dorm in '97, if you get a few people streaming different files off your computer at the same time the computer ground to a halt. Ah, the kind of problem I ran into back in the good old days of network sharing mp3's in my dorm in '97, if you get a few people streaming different files off your computer at the same time the computer ground to a halt.

]]> by: Ice8205 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4213 Tue, 03 Feb 2009 03:04:11 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4213 This is the difference between file servers and home computers, and why UPS's are so important on file servers. The average home computer has only a few files in memory and not written to disk. And likely not all that important if the power fails and stuff doesn't get written. File servers, on the other hand, generally have lots of imporant stuff still in memory and not written to disk. This is the difference between file servers and home computers, and why UPS's are so important on file servers.

The average home computer has only a few files in memory and not written to disk. And likely not all that important if the power fails and stuff doesn't get written.

File servers, on the other hand, generally have lots of imporant stuff still in memory and not written to disk.

]]> by: Daniel Rutter http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4212 Mon, 02 Feb 2009 23:15:09 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4212 One of you bastard readers reminded me, via e-mail, that I really ought to run <a href="http://en.wikipedia.org/wiki/Chkdsk">chkdsk</a> over every drive in the computer now. So I did that, at eight o'clock in the morning. At eleven o'clock at night, it finished! And I wasn't even doing every disk. A couple weren't locked, so I chkdsked them in the background while doing other stuff. Four disks remained that had to be done on the next restart, though, and those four were apparently chkdsked in a process involving telegrams and sea mail. It got to the point where I was checking to see if the damn thing was in a loop. Still, it's one of those better-out-than-in things. One of you bastard readers reminded me, via e-mail, that I really ought to run chkdsk over every drive in the computer now.

So I did that, at eight o'clock in the morning.

At eleven o'clock at night, it finished!

And I wasn't even doing every disk. A couple weren't locked, so I chkdsked them in the background while doing other stuff. Four disks remained that had to be done on the next restart, though, and those four were apparently chkdsked in a process involving telegrams and sea mail. It got to the point where I was checking to see if the damn thing was in a loop.

Still, it's one of those better-out-than-in things.

]]> by: jmguazzo http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4211 Mon, 02 Feb 2009 20:54:05 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4211 I don't use terracopy, I use Total Commander. It has a "Queue" instruction when copying or moving. I used it for moving very larges files and never had any problem. And you can even specify buffer size,Pause it, or limit copy's "speed". The best about TC, it looks like nc in dos or mc in linux... Btw Dan, as I learned in woodworking classes, "Always blame the tool." I don't use terracopy, I use Total Commander. It has a "Queue" instruction when copying or moving. I used it for moving very larges files and never had any problem.
And you can even specify buffer size,Pause it, or limit copy's "speed".
The best about TC, it looks like nc in dos or mc in linux...

Btw Dan, as I learned in woodworking classes, "Always blame the tool."

]]> by: phrantic http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4210 Mon, 02 Feb 2009 20:25:12 +0000 http://dansdata.blogsome.com/2009/02/01/the-copy-of-doom/#comment-4210 "The identity of the Doom-Copier will remain a secret, to protect him or her from violent reprisals." It was one of the cats, wasn't it? "The identity of the Doom-Copier will remain a secret, to protect him or her from violent reprisals."

It was one of the cats, wasn't it?

]]>