Current version

v1.10.4 (stable)

Navigation

Main page
Archived news
Downloads
Documentation
   Capture
   Compiling
   Processing
   Crashes
Features
Filters
Plugin SDK
Knowledge base
Donate
Contact info
Forum
 
Other projects
   Altirra

Search

Archives

01 Dec - 31 Dec 2013
01 Oct - 31 Oct 2013
01 Aug - 31 Aug 2013
01 May - 31 May 2013
01 Mar - 31 Mar 2013
01 Feb - 29 Feb 2013
01 Dec - 31 Dec 2012
01 Nov - 30 Nov 2012
01 Oct - 31 Oct 2012
01 Sep - 30 Sep 2012
01 Aug - 31 Aug 2012
01 June - 30 June 2012
01 May - 31 May 2012
01 Apr - 30 Apr 2012
01 Dec - 31 Dec 2011
01 Nov - 30 Nov 2011
01 Oct - 31 Oct 2011
01 Sep - 30 Sep 2011
01 Aug - 31 Aug 2011
01 Jul - 31 Jul 2011
01 June - 30 June 2011
01 May - 31 May 2011
01 Apr - 30 Apr 2011
01 Mar - 31 Mar 2011
01 Feb - 29 Feb 2011
01 Jan - 31 Jan 2011
01 Dec - 31 Dec 2010
01 Nov - 30 Nov 2010
01 Oct - 31 Oct 2010
01 Sep - 30 Sep 2010
01 Aug - 31 Aug 2010
01 Jul - 31 Jul 2010
01 June - 30 June 2010
01 May - 31 May 2010
01 Apr - 30 Apr 2010
01 Mar - 31 Mar 2010
01 Feb - 29 Feb 2010
01 Jan - 31 Jan 2010
01 Dec - 31 Dec 2009
01 Nov - 30 Nov 2009
01 Oct - 31 Oct 2009
01 Sep - 30 Sep 2009
01 Aug - 31 Aug 2009
01 Jul - 31 Jul 2009
01 June - 30 June 2009
01 May - 31 May 2009
01 Apr - 30 Apr 2009
01 Mar - 31 Mar 2009
01 Feb - 29 Feb 2009
01 Jan - 31 Jan 2009
01 Dec - 31 Dec 2008
01 Nov - 30 Nov 2008
01 Oct - 31 Oct 2008
01 Sep - 30 Sep 2008
01 Aug - 31 Aug 2008
01 Jul - 31 Jul 2008
01 June - 30 June 2008
01 May - 31 May 2008
01 Apr - 30 Apr 2008
01 Mar - 31 Mar 2008
01 Feb - 29 Feb 2008
01 Jan - 31 Jan 2008
01 Dec - 31 Dec 2007
01 Nov - 30 Nov 2007
01 Oct - 31 Oct 2007
01 Sep - 30 Sep 2007
01 Aug - 31 Aug 2007
01 Jul - 31 Jul 2007
01 June - 30 June 2007
01 May - 31 May 2007
01 Apr - 30 Apr 2007
01 Mar - 31 Mar 2007
01 Feb - 29 Feb 2007
01 Jan - 31 Jan 2007
01 Dec - 31 Dec 2006
01 Nov - 30 Nov 2006
01 Oct - 31 Oct 2006
01 Sep - 30 Sep 2006
01 Aug - 31 Aug 2006
01 Jul - 31 Jul 2006
01 June - 30 June 2006
01 May - 31 May 2006
01 Apr - 30 Apr 2006
01 Mar - 31 Mar 2006
01 Feb - 29 Feb 2006
01 Jan - 31 Jan 2006
01 Dec - 31 Dec 2005
01 Nov - 30 Nov 2005
01 Oct - 31 Oct 2005
01 Sep - 30 Sep 2005
01 Aug - 31 Aug 2005
01 Jul - 31 Jul 2005
01 June - 30 June 2005
01 May - 31 May 2005
01 Apr - 30 Apr 2005
01 Mar - 31 Mar 2005
01 Feb - 29 Feb 2005
01 Jan - 31 Jan 2005
01 Dec - 31 Dec 2004
01 Nov - 30 Nov 2004
01 Oct - 31 Oct 2004
01 Sep - 30 Sep 2004
01 Aug - 31 Aug 2004

Stuff

Powered by Pivot  
XML: RSS feed 
XML: Atom feed 

§ How not to write backup software

I bought a Seagate 160GB external USB2 hard drive yesterday for the purpose of backing up my laptop's 60GB hard drive. Most of the stuff on my hard drive is junk not work saving often in very large files called test.avi but I have a lot of source code that's worth keeping, and up to this point, hasn't been backed up in any reliable form except for a few dozen archives on SourceForge. Recently I thought I should be a little less cavalier about my data and actually try a backup solution.

The drive came with a backup program called BounceBack Express. I ordinarily take a rather dim view of software that comes with hardware; the drivers are usually out of date and the supplemental software is usually a crippled version. This was no different, in that the BounceBack Express software is a reduced-functionality version of the BounceBack backup program. I decided to try it out anyway, though, just to see how good (or bad) it was. The backup drive was more than twice as large as the source disk anyway, so backing up junk wasn't going to be a big deal. So I just decided to just let it back up the whole drive on default settings.

With the verification option on.

Verification is a simple feature to implement. The idea is that once you've backed up data onto the drive, you read back the written data to make sure it matches the source to make sure the backup is good. After all, the worst that can happen is that the primary drive fails, you try the backup, and then realize that the backup wasn't made properly and is useless. The BounceBack Express software re-reads the backup file after writing each one, comparing it against the original.

Except that when it finished copying a 300MB file and began its verification pass, the backup drive's activity light went off.

I have a lot of memory in my laptop, 1.2GB, most of which is not used directly by running programs. As soon as I saw that activity light go off I immediately got the sinking feeling that the makers of the backup software had made of the most classic errors in verifying a file copy on Windows. The next time I saw a large file get transferred, I launched Sysinternals FileMon, and dumped a trace of the verify pass (edited for brevity):

11877   FASTIO_READ   C:...BigFile  Offset: 1245184 Length: 65536
11878   FASTIO_READ   F:...BigFile  Offset: 1245184 Length: 65536
11879   FASTIO_READ   C:...BigFile  Offset: 1310720 Length: 65536
11880   FASTIO_READ   F:...BigFile  Offset: 1310720 Length: 65536
11881   IRP_MJ_READ*  C:...BigFile  Offset: 1376256 Length: 65536
11882   IRP_MJ_READ*  C:...BigFile  Offset: 1441792 Length: 65536
11883   FASTIO_READ   C:...BigFile  Offset: 1376256 Length: 65536
11884   FASTIO_READ   F:...BigFile  Offset: 1376256 Length: 65536
11885   FASTIO_READ   C:...BigFile  Offset: 1441792 Length: 65536
11886   FASTIO_READ   F:...BigFile  Offset: 1441792 Length: 65536
11887   IRP_MJ_READ*  C:...BigFile  Offset: 1507328 Length: 65536
11888   IRP_MJ_READ*  C:...BigFile  Offset: 1572864 Length: 65536
11889   FASTIO_READ   C:...BigFile  Offset: 1507328 Length: 65536

Notice that the reads from the target F: drive only have FASTIO_READ requests associated with then, not IRP_MJ_READs like the C: reads. Reading a Windows IT Pro article by Mark Russinovich confirmed my suspicions: a FASTIO_READ request can only directly satisfy a read request if the data is in the disk cache, and an IRP is required otherwise. Which means...

...the backup software wasn't actually verifying anything because it used buffered I/O and was re-reading the entire file from the Windows disk cache.

I don't think I'll be purchasing the full-version upgrade of this software.

Comments

Comments posted:


Why not to use robotcopy, it is command line, from microsoft resouce kit

hexboy - 21 08 05 - 21:44


I use Norton Ghost for almost all my backup jobs. It reads the entire disk, including partition information and dumps it in a big file on your external HD. Much faster than copying single files. You can even browse the archive, extract files, do incremental backups. I found it 10 times easier to use and much more reliable than any other backup method.

I know it sounds like a sales pitch, but I was really amazed that you could get something where someone had actually THOUGHT about how it could done easy, fast and reliable.

sh0dan (link) - 22 08 05 - 04:29


Agreed. Ghost works. 5 minutes to restore a 4GB partition is nice. Plus as said, you can browse inside it.
It also compresses.

SonOfAdam - 22 08 05 - 04:47


I use a damn small program called drivesnapshot. No install, exe just runs from windows and dos. Is also integrated in partPE windows bootdisk. Totaly worth buying. I am not affiliated with them, I just like the program. http://www.drivesnapshot.de

Hannes Dorn - 22 08 05 - 09:03


PowerQuest Drive Image does the same. Best used from DOS, though.

Murmel - 22 08 05 - 13:49


As folks have posted alternatives, while it has it's shortcomings Nero backup suits me. Not a replacement for Ghost, directory structure is preserved & files are essentially zipped so it's easier sometimes to just browse a backup and grab a file or three.

mike - 22 08 05 - 16:52


The problem with Ghost is that it can't handle dynamic disks properly. It can read them fine, but it creates partitions as a basic disk. Typically people want dynamic disks to be able to expand volumes in the future. Plug in new RAID disk, expand the array, reboot, expand the volume. However, such expansion is only possible on volumes that were created AFTER the disk is made dynamic. Thus, you have to make the dynamic volumes on an existing installation of Windows and restore the Ghost images to the new volumes.
I keep meaning to find out if BartPE can make dynamic disks...

Michael - 22 08 05 - 19:22


I've noticed essentially the same problem when burning CDs. I acquired a new DVD burner this year which included a new version of Nero. My old (about a year older) CD burner with the previous version of Nero always ejected the CD tray before verifying the data, which I suppose caused the system to invalidate the disk cache. The new drive, however, does not eject the disk between burning and verifying. So when I make a CD with the DVD drive, the data remains in cache and Nero doesn't read it off the disk. Of course, when burning a DVD, there is too much data for the cache, so it has to re-read the entire disc.

And yes, I do know that it's the drive and not just the new version of Nero, since the old CD burner (which I haven't removed) still ejects before verifying.

The point here is that this problem isn't just in backup software.

Quadro - 23 08 05 - 02:11


My personal program of choice for backups is the humble xcopy. I have a task scheduled to run when the computer's been idle for 5 minutes, but no more frequently than once every two hours. This task simply runs "cmd.exe /C XCOPY src dest /M /D /S /E /V /C /I /H /R /K /O /X /Y", and copies all changed data from one hard disk to the other. I probably don't need all of those switches to get the required result, but it does the trick. It is scheduled to run as often as it does as it may not get to run for several days, depending on computer usage and when I hibernate the computer.

For your laptop backing up to an external hard disk, you could add an "IF EXIST" statement to check if the disk is physically present. It might be a good idea to assign a different drive letter to the external disk (something like M:) to avoid problems when you have a different disk or memory stick plugged in (drive letter assignments are only sticky if you explicitly set them, else Windows uses the first available letter).

BoggyB (link) - 23 08 05 - 06:05


BTW, the comment counter (at the end of each blog entry) is broken. It only shows "%num% comments" instead of the actual number of comments.

BoggyB (link) - 23 08 05 - 06:07


Someone who have the same experience as Avery should start an OpenSource project for backup software. I've tried some expensive backup software for DAT and VXA's. None of them really can convince me to pay ... complex GUI and fancy icons everywhere, but the real backup functions are rough and not easy.

VirtualDub do have a lot of functions, and glad it's easy to use and rarely crash.

johnsonlam (link) - 23 08 05 - 08:28


If you're looking for free and simple backup software for simply transferring files every now and then I have found that 2brightsparks syncback also works very well. I use it to copy the contents of one removable harddrive to another. The nice thing here is that you can choose to have files removed from the backup drive that are no longer present on the main drive.

Songsoverruins - 24 08 05 - 06:12


Amazing, 30 different people responded and there were 30 different programs that they suggested to use. I will agree with the second post, Ghost is nice and that's what I use.
Cheers,

Landi (link) - 25 08 05 - 20:52


I am using the free version of SyncBackup from 2BrightSparks and i like this tool a lot. Actually, the free version has so many useful features i never had time to test yet, i really wonder what the payware version SyncBackSE can offer in addition to that ?

Christian
matroska project admin
http://www.matroska.org

ChristianHJW (link) - 28 08 05 - 16:31


I use VDUB for backups, just put an AVI Header to the HD and compress it all to XVID, you can even lower Audio Volume to make Drive more quiet! :)
Now seriously, I always have HDs in FAT32 (excepting last NTFS partition in upper >137Gb area); and I have an operating Win98 in same WinXP C: partition, I backup from win98 using TotalCommander's Zip Feature: Create One Zip per Directory. The reason I use 98 is for avoiding XP locking system archives that can't be backup otherwise, besides, from there I can repair any missing file, clean virus, etc
For automatic regular DOCs backups after analysing at least ten different programs, I choose argentum backup http://www.argentuma.com (I'm not affiliated to them), it was the only to do all what I wanted

isidro - 29 08 05 - 00:30


I've used Cobian Backup for a year now and I've been happy with it. It's a little sensitive, but it works well at backing up my pst files on a routine basis. Worth the price (i.e. free).

Staff Monkey (link) - 31 08 05 - 20:34


For a full disc backup check out PING which uses a bootable Linux CD to run Partimage. Since you're operating off the CD, your Windows drive is just another data disc.

ken - 11 12 07 - 07:21

Comment form


Please keep comments on-topic for this entry. If you have unrelated comments about VirtualDub, the forum is a better place to post them.
Name:  
Remember personal info?

Email (Optional):
Your email address is only revealed to the blog owner and is not shown to the public.
URL (Optional):
Comment: /

An authentication dialog may appear when you click Post Comment. Simply type in "post" as the user and "now" as the password. I have had to do this to stop automated comment spam.



Small print: All html tags except <b> and <i> will be removed from your comment. You can make links by just typing the url or mail-address.