Thanks to a lucky Dr. Watson dump sent in by a user, I've finally tracked down the reason that recent versions of VirtualDub crash on startup on some machines running Windows 98/ME. This was impossible for me to track down for a long time since I couldn't reproduce the problem myself and the crash occurred in a 16-bit DLL. I didn't realize that Dr. Watson on Windows 98 is able to do a stack trace across both 16-bit and 32-bit calls with disassembly, which make it MUCH easier to track down what was going wrong. Because of the severity of the problem, I've fast-tracked the 1.6.19 release.
Build 24478 (1.6.19, stable): [June 3, 2007] [bugs fixed] * Fixed crash when creating batch jobs with certain Pinnacle video codecs. * Fixed crash when reading top-down BMP files. * Added workaround for crash on startup on some Windows 98/ME machines.
This isn't a 100% fix, because I've been able to reproduce the problem in ways that seem unfixable at the application level, but at least one person has reported the problem fixed, so I figured it was worthy of release anyway. Details of the cause and fix after the jump.
The crash turned out to be triggered by a call to the USER32 GetAncestor() function, which I use within my main message loop to determine the top level window for each input event. This is necessary to support accelerators (keyboard shortcuts) in dialogs -- without it, keystrokes can be trapped by controls inside of a dialog and never tested against the accelerator table. Therefore, I use GetAncestor() to retrieve the top-level window, look up if that window has an accelerator table assocated with it, and call TranslateAccelerator() if it does. GetAncestor() on the user's machine was actually crashing within USER32 itself.
If you look at the MSDN library documentation for CreateWindow(), it says that you can specify a parent window handle of HWND_MESSAGE to create a "message-only window." This is a window whose sole purpose is to receive messages, never to be displayed or to receive input. I suppose this hack was put in because too many people were creating such windows, given the lack of support for message targets that aren't threads or windows. Now, the documentation says that HWND_MESSAGE is valid only in Windows 2000/XP or newer, and <windows.h> will only define this constant if you have WINVER set to at least 0x0500. In reality, the call succeeds just fine under Windows 98 you get a window that can receive messages and whose window procedure is invoked as usual.
Until someone calls GetAncestor() on your window.
A message-only window doesn't show up in Spy++'s list, but if you call IsWindow() on it, it returns TRUE. Call GetAncestor(hwnd, GA_ROOT), however, and USER32 dies, turns over, and smells bad in 16-bit code. The workaround for this is to use GetParent(), which isn't fooled by the odd window and returns NULL instead. This is what I did in 1.6.19, and it solved the user's startup problem. The reason I say that this isn't a 100% fix, though, is that when I wrote a repro case for this, I originally created a message only window in a DLL and then started a 500ms timer on it. Not only did this cause a crash in DialogBoxParamA() which I can't really work around, since it's Microsoft's dialog message loop but it once caused several blue screens, a white Windows 3.1-style General Application Fault dialog, and finally took down the whole OS. The best that I can tell is that this is a problem in the core window manager, and is something that only Microsoft could truly fix... but given the support status of Windows 98/ME, it probably won't ever be fixed.
Windows XP doesn't seem to have this problem, as GetAncestor(hwnd, GA_ROOT) just returns NULL. It seems that Windows 2000 would originally blue screen in win32k.sys under similar situations, but the problem has been fixed in Service Pack 3.
While I still believe in supporting the Windows 9x line which is why VirtualDub 1.6.x still supports Windows 95 and above and 1.7.x supports Windows 98 and above the Windows programming world will be a bit nicer once these operating systems truly become ignorable. Many new applications are already limiting themselves to Windows XP and up, but even the .NET Framework still supports running under 98 to some extent.