§ ¶How not to write backup software
I bought a Seagate 160GB external USB2 hard drive yesterday for the purpose of backing up my laptop's 60GB hard drive. Most of the stuff on my hard drive is junk not work saving — often in very large files called test.avi — but I have a lot of source code that's worth keeping, and up to this point, hasn't been backed up in any reliable form except for a few dozen archives on SourceForge. Recently I thought I should be a little less cavalier about my data and actually try a backup solution.
The drive came with a backup program called BounceBack Express. I ordinarily take a rather dim view of software that comes with hardware; the drivers are usually out of date and the supplemental software is usually a crippled version. This was no different, in that the BounceBack Express software is a reduced-functionality version of the BounceBack backup program. I decided to try it out anyway, though, just to see how good (or bad) it was. The backup drive was more than twice as large as the source disk anyway, so backing up junk wasn't going to be a big deal. So I just decided to just let it back up the whole drive on default settings.
With the verification option on.
Verification is a simple feature to implement. The idea is that once you've backed up data onto the drive, you read back the written data to make sure it matches the source to make sure the backup is good. After all, the worst that can happen is that the primary drive fails, you try the backup, and then realize that the backup wasn't made properly and is useless. The BounceBack Express software re-reads the backup file after writing each one, comparing it against the original.
Except that when it finished copying a 300MB file and began its verification pass, the backup drive's activity light went off.
I have a lot of memory in my laptop, 1.2GB, most of which is not used directly by running programs. As soon as I saw that activity light go off I immediately got the sinking feeling that the makers of the backup software had made of the most classic errors in verifying a file copy on Windows. The next time I saw a large file get transferred, I launched Sysinternals FileMon, and dumped a trace of the verify pass (edited for brevity):
11877 FASTIO_READ C:...BigFile Offset: 1245184 Length: 65536
11878 FASTIO_READ F:...BigFile Offset: 1245184 Length: 65536
11879 FASTIO_READ C:...BigFile Offset: 1310720 Length: 65536
11880 FASTIO_READ F:...BigFile Offset: 1310720 Length: 65536
11881 IRP_MJ_READ* C:...BigFile Offset: 1376256 Length: 65536
11882 IRP_MJ_READ* C:...BigFile Offset: 1441792 Length: 65536
11883 FASTIO_READ C:...BigFile Offset: 1376256 Length: 65536
11884 FASTIO_READ F:...BigFile Offset: 1376256 Length: 65536
11885 FASTIO_READ C:...BigFile Offset: 1441792 Length: 65536
11886 FASTIO_READ F:...BigFile Offset: 1441792 Length: 65536
11887 IRP_MJ_READ* C:...BigFile Offset: 1507328 Length: 65536
11888 IRP_MJ_READ* C:...BigFile Offset: 1572864 Length: 65536
11889 FASTIO_READ C:...BigFile Offset: 1507328 Length: 65536
Notice that the reads from the target F: drive only have FASTIO_READ requests associated with then, not IRP_MJ_READs like the C: reads. Reading a Windows IT Pro article by Mark Russinovich confirmed my suspicions: a FASTIO_READ request can only directly satisfy a read request if the data is in the disk cache, and an IRP is required otherwise. Which means...
...the backup software wasn't actually verifying anything because it used buffered I/O and was re-reading the entire file from the Windows disk cache.
I don't think I'll be purchasing the full-version upgrade of this software.
Why not to use robotcopy, it is command line, from microsoft resouce kit
hexboy - 21 08 05 - 21:44
I use Norton Ghost for almost all my backup jobs. It reads the entire disk, including partition information and dumps it in a big file on your external HD. Much faster than copying single files. You can even browse the archive, extract files, do incremental backups. I found it 10 times easier to use and much more reliable than any other backup method.
I know it sounds like a sales pitch, but I was really amazed that you could get something where someone had actually THOUGHT about how it could done easy, fast and reliable.
sh0dan (link) - 22 08 05 - 04:29
Agreed. Ghost works. 5 minutes to restore a 4GB partition is nice. Plus as said, you can browse inside it.
It also compresses.
SonOfAdam - 22 08 05 - 04:47
I use a damn small program called drivesnapshot. No install, exe just runs from windows and dos. Is also integrated in partPE windows bootdisk. Totaly worth buying. I am not affiliated with them, I just like the program. http://www.drivesnapshot.de
Hannes Dorn - 22 08 05 - 09:03
PowerQuest Drive Image does the same. Best used from DOS, though.
Murmel - 22 08 05 - 13:49
As folks have posted alternatives, while it has it's shortcomings Nero backup suits me. Not a replacement for Ghost, directory structure is preserved & files are essentially zipped so it's easier sometimes to just browse a backup and grab a file or three.
mike - 22 08 05 - 16:52
The problem with Ghost is that it can't handle dynamic disks properly. It can read them fine, but it creates partitions as a basic disk. Typically people want dynamic disks to be able to expand volumes in the future. Plug in new RAID disk, expand the array, reboot, expand the volume. However, such expansion is only possible on volumes that were created AFTER the disk is made dynamic. Thus, you have to make the dynamic volumes on an existing installation of Windows and restore the Ghost images to the new volumes.
I keep meaning to find out if BartPE can make dynamic disks...
Michael - 22 08 05 - 19:22
I've noticed essentially the same problem when burning CDs. I acquired a new DVD burner this year which included a new version of Nero. My old (about a year older) CD burner with the previous version of Nero always ejected the CD tray before verifying the data, which I suppose caused the system to invalidate the disk cache. The new drive, however, does not eject the disk between burning and verifying. So when I make a CD with the DVD drive, the data remains in cache and Nero doesn't read it off the disk. Of course, when burning a DVD, there is too much data for the cache, so it has to re-read the entire disc.
And yes, I do know that it's the drive and not just the new version of Nero, since the old CD burner (which I haven't removed) still ejects before verifying.
The point here is that this problem isn't just in backup software.
Quadro - 23 08 05 - 02:11
My personal program of choice for backups is the humble xcopy. I have a task scheduled to run when the computer's been idle for 5 minutes, but no more frequently than once every two hours. This task simply runs "cmd.exe /C XCOPY src dest /M /D /S /E /V /C /I /H /R /K /O /X /Y", and copies all changed data from one hard disk to the other. I probably don't need all of those switches to get the required result, but it does the trick. It is scheduled to run as often as it does as it may not get to run for several days, depending on computer usage and when I hibernate the computer.
For your laptop backing up to an external hard disk, you could add an "IF EXIST" statement to check if the disk is physically present. It might be a good idea to assign a different drive letter to the external disk (something like M:) to avoid problems when you have a different disk or memory stick plugged in (drive letter assignments are only sticky if you explicitly set them, else Windows uses the first available letter).
BoggyB (link) - 23 08 05 - 06:05
BTW, the comment counter (at the end of each blog entry) is broken. It only shows "%num% comments" instead of the actual number of comments.
BoggyB (link) - 23 08 05 - 06:07
Someone who have the same experience as Avery should start an OpenSource project for backup software. I've tried some expensive backup software for DAT and VXA's. None of them really can convince me to pay ... complex GUI and fancy icons everywhere, but the real backup functions are rough and not easy.
VirtualDub do have a lot of functions, and glad it's easy to use and rarely crash.
johnsonlam (link) - 23 08 05 - 08:28
If you're looking for free and simple backup software for simply transferring files every now and then I have found that 2brightsparks syncback also works very well. I use it to copy the contents of one removable harddrive to another. The nice thing here is that you can choose to have files removed from the backup drive that are no longer present on the main drive.
Songsoverruins - 24 08 05 - 06:12
Amazing, 30 different people responded and there were 30 different programs that they suggested to use. I will agree with the second post, Ghost is nice and that's what I use.
Landi (link) - 25 08 05 - 20:52
I am using the free version of SyncBackup from 2BrightSparks and i like this tool a lot. Actually, the free version has so many useful features i never had time to test yet, i really wonder what the payware version SyncBackSE can offer in addition to that ?
matroska project admin
ChristianHJW (link) - 28 08 05 - 16:31
I use VDUB for backups, just put an AVI Header to the HD and compress it all to XVID, you can even lower Audio Volume to make Drive more quiet! :)
Now seriously, I always have HDs in FAT32 (excepting last NTFS partition in upper >137Gb area); and I have an operating Win98 in same WinXP C: partition, I backup from win98 using TotalCommander's Zip Feature: Create One Zip per Directory. The reason I use 98 is for avoiding XP locking system archives that can't be backup otherwise, besides, from there I can repair any missing file, clean virus, etc
For automatic regular DOCs backups after analysing at least ten different programs, I choose argentum backup http://www.argentuma.com
(I'm not affiliated to them), it was the only to do all what I wanted
isidro - 29 08 05 - 00:30
I've used Cobian Backup for a year now and I've been happy with it. It's a little sensitive, but it works well at backing up my pst files on a routine basis. Worth the price (i.e. free).
Staff Monkey (link) - 31 08 05 - 20:34
For a full disc backup check out PING which uses a bootable Linux CD to run Partimage. Since you're operating off the CD, your Windows drive is just another data disc.
ken - 11 12 07 - 07:21