§ ¶VirtualDub 1.7.5 released

Quick shot: 1.7.5 is out. I fixed a few bugs reported past 1.7.4 -- one of the problems with having experimental/stable split releases is that not as many people test the experimental releases, so I often get a bunch of reports after a branch goes stable. The other problem was that I had forgotten to integrate from the 1.6.x branch, so the fixes that went into 1.6.19 hadn't made it into 1.7.4. Oops.

Changelist after the jump.

(Read more....)

§ ¶How to fix debugger visualizers for VS2005 SP1 and VS2008

If you've written a bunch of custom visualizers for the Visual Studio debugger, you may have run into the problem lately of the preview() section of your visualizer causing the debugger to hang or crash, or simply displaying ... when used in nested visualizers. I tracked this down to a change that was snuck into VS2005 SP1 and which is also in VS2008 beta 2. It used to be the case that $c (container) and $e (element) were equivalent in the preview() block. This is no longer the case -- $c instead refers to the top level container. This means that you can run into problems if you have a nested setup like this:

A {
    preview($c.m)
}
B {
    preview($c.m)
}

Formerly, this would work fine with an object of type B which derives from A. In VS2005 SP1 / VS2008b2, this causes the debugger to crash, because instead of evaluating B::m and then A::m, it evaluates B::m over and over and then blows up. In visualizers where the fields of the nested type don't alias with the type that started the evaluation, you'll see ... instead as that's the debugger's way of signifying evaluation error.

In the cases that I've seen, replacing $c with $e within preview() fixes the problem.

I also have some additional tips that I've learned since writing the original blog entry:

• You can avoid bumping the indices of the auto-numbered elements by surrounding named elements in #(), which will push them after the auto-numbered ones:
  

  MyVector {
      children(
          #(
              #array(expr: $c.s[$i], size: $c.len),
              #(raw: [$c,!])
          )
      )
  }
  

  
  This can also be used as a dirty way to force w to the end of the variable list for a 4-vector that has x, y, z, and w components, which would ordinarily be sorted alphabetically.
  
  (Although this bug still exists in VS2005 SP1, it appears to be fixed in VS2008 beta 2 — the indices start at [0] regardless.)
  
• The expression parser is whitespace sensitive when parsing template type expressions, so you should mimic the debugger exactly. Generally, this means the unusual style of placing a space before the comma separating template elements and not after, i.e.: mytype<$T1 ,$T2>.
  
• The "b" at the end of a debugger format tag means "bare" and is useful for displaying a value without adornment like quotes and 0x prefixes. It's used in the existing visualizers for strings (foo,sub), but you can also use it for integers: value,xb
  
• If you use #array() within a preview block, the debugger will automatically insert commas between each element.

(Read more....)

§ ¶What's new in the 1.7 series

A reader wanted to a summary of all the new features in the VirtualDub 1.7 series, so....

Improved file format support

• Maya IFF (read).
• PNG (now read/write).
• Animated GIF (read/write).
• Adobe Filmstrip (read/write).
• WAVE64 (read/write): Now break the 2GB barrier for audio files.
• Raw audio (write).
• IFF ANIM: Added support for HAM, Halfbrite, and AGA modes.
• AVI: Direct YV24 and Y800 support.
• Input driver plugin API: Add support for custom formats.

Enhanced editing

• Smart rendering: Recompress only the portions of the video you've edited and copy the rest at direct-mode speeds.
• Audio display: View the waveform or spectrogram of an uncompressed audio stream. Visually drag an audio stream back into sync.
• On-the-fly compressed preview: View the post-compression output for a moment. Toggle it back off without speed penalty.
• Preserve empty frames: Process a "120 fps" video at actual frame rate.

More powerful filtering

• Revamped resize filter UI: Easy relative sizing and aspect ratio preservation. Save your favorite settings.
• New center cut algorithm for better center/side audio separation.
• Cleaner pitch scaling algorithm.
• Warp resize: Cartoon-friendly resizing.
• Filter curves: Fade a filter in and out over time. Combine with smart rendering to direct-copy the parts of the video you don't filter.

Improved compatibility

• Fixed display issues when running under Windows Vista.
• Recover broken AVI files written by Vista RTM. (Note: There is now a patch available from Microsoft for this issue.)
• Workarounds for broken JPEGs and mu-law audio streams written by digital cameras.
• Resolved issues with various drivers and codecs, including the Adaptec GameBridge, Pinnacle DV codec, and Blackmagic MJPEG codec.
• No more dangling frames when writing segmented AVIs while compressing video.

Better display support

• Better vertical sync and field display support: See interlaced video more faithfully.
• Improved D3DFX annotation/semantics support: Prototype more powerful algorithms.
• Direct3D accelerated color conversion for UYVY, YUY2, YV24, YV16, I420, and YVU9 formats: Preview faster under Vista with WDDM drivers.

Expanded video capture capabilities

• Video proc amp support: Easy access to a device's brightness, contrast, saturation, and hue controls.
• Configurable global stop/start hotkeys.
• Tuner antenna/cable switching.
• More settings auto-save.
• Improved audio resync code.

Stability, stability, stability

• Lots of bug fixes that were too risky to push into the 1.6.x series.

(Read more....)

§ ¶VirtualDub 1.7.4 released

VirtualDub 1.7.4 is out -- basically the same as 1.7.3, except for a few bug fixes. This is also the first release of the 1.7.x series marked as "stable," replacing 1.6.x in that position. As such, this is basically an end-of-support announcement for VirtualDub running on Windows 95 -- from now on I will likely only be making changes and fixes to the 1.7.x series, which requires Windows 98.

I haven't decided when to start up experimental releases again, but I don't think I'll be starting up 1.8.x just yet... not that version numbers matter that much. I'll probably be posting some bleeding-edge releases on the forum, though, if you're so inclined.

Changelist after the jump.

(Read more....)

§ ¶You might be a developer if....

My video playback application started crashing randomly on me a couple of days ago. I had initially chalked it up to a dodgy video files, since I'm rather used to having video decoders crash on marginal data. However, after it kept happening on some known-good files, I attached a debugger and broke into the crashed application.

And then promptly swore when I saw d3d9d on the top of the call stack.

See, I occasionally do a bit of 3D graphics, and when doing so, I set Direct3D to run with the debug runtime and the "break on D3D error" setting enabled. This is great when doing D3D development, but not so good when running a video player that uses 3D to display video and occasionally makes some minor transgressions. It hasn't occurred to me that the setting I was tweaking was also screwing over the video player parked in the corner of the screen.

Of course, this isn't the first time I've done this -- Titan Quest went from painfully slow to only moderately slow once I'd realized my mistake. I also had something similar happen when I had forgotten that I'd installed the DirectShow debug runtime and couldn't figure out where the timestamp display in the middle of the video player was coming from.

(Read more....)

§ ¶Making a file format standard is hard work

There has been a lot of discussion lately over Microsoft's Open Office XML (OOXML) format and how it has been going through the ISO standardization process. Now, I'm not in the business of writing productivity software nor do I have any interest in doing so, but purely from a technical standpoint -- political issues aside -- I'd have to agree with detractors that OOXML is not a good standard. Underspecification bothers me the most. Not adequately specifying part of a standard results in ambiguity that can kill the utility of parts of a file format; once everyone starts making different errors in writing the same field, it can become impossible to discern the correct meaning. Having a tag such as "useWord97LineBreak" in a standard without an actual description of what Word 97 does is, of course, an egregious offense. However, I will say that trying to fully specify a file format isn't easy, and OOXML definitely wouldn't have been the first ISO standard that suffers from holes.

The reason is that writing a file format is, well, hard.

Let's take the simple example of storing a frame rate in a file. Because common frame rates are non-integral, and we want to maintain accuracy even over long durations, we'll store it in rational form, as the ratio of two 32-bit integers:

uint32_t frameRateNumerator;
uint32_t frameRateDenominator;

(This is, in fact, how AVI stores frame rates. It is also used with Direct3D 10.)

How many issues can arise with these two fields? Well:

• Are there minimums and maximums to the stored fraction? Are there certain profiles that can rely on restricted values, such as for mobile devices?
• Are there recommended values for common frame rates? (These can double as compliance tests.)
• Can the numerator be zero? This would mean a frame rate of zero.
• Can the denominator be zero? You can't divide by zero. What does it mean?
• If zero in either field is invalid, what should programs do? Should they reject it, automatically correct it to some value, or is it up to the implementation?
• What is the byte order of these fields, little-endian or big-endian?
• Must the stored fraction be stored in lowest terms? Is there any significance if they are not, and should an implementation reduce an unnormalized fraction? What algorithm is recommended for reducing fractions? (Finding one was a bit harder when you had to go to the library instead of doing a web search.)
• If an application approximates the fraction to a single value, what is the minimum recommended or required precision? Are there specific values that must always be represented exactly?
• Do these fields need to be consistent with other fields in the file? For instance, are there times when the same frame rate shows up multiple times in the file? If they are different, how are they reconciled?

There are a number of bad outcomes that can arise from not answering these questions. One possibility is that applications commonly write 30/1 for NTSC and then interpret that on read as NTSC, even though NTSC is actually 29.97. Another possibility is that an application writes garbage into the frame rate fields and then ignores the values on read, because it works in a medium that already has a defined frame rate and not all programs validate or use the value on read. A third possibility is that everyone assumes the order is backwards and the odd program written by the person who actually reads the spec can't read everyone else's files. And yes, I've seen all of these kinds of mischief before.

Good file formats are rare, but in my opinion, the Portable Network Graphics (PNG) specification is among the better ones. It uses clear language (must/must not/should/should not), it has rationales for various design decisions, and it attempts to advise what to do when dealing with non-compliance. For instance, when talking about converting samples between different bit depths, it describes the best case (linear mapping), an acceptable approximation (bit replication), and says what you should not do and why (bit shift left). That level of detail doesn't prevent all accidents, but at least it reduces them through awareness, and clarifies who is at fault when an interoperability problem occurs.

(Read more....)

§ ¶Website go splat

Some of you might have noticed that this website went splat for a while today. Well, that happened because my web host (pair) updated a bunch of stuff on the server including FreeBSD and Apache. I knew it was coming and figured there might be some issues, but I didn't expect the entire website to go down. Turns out the problem had to do with my use of MultiViews, which I was using for transparent gzip compression. I wasn't able to fix the problem, even with the recommended tweaks, so I just ended up ditching MultiViews and hacked out a mod_rewrite based solution. Well, eventually. A beneficial side effect of my frustration is that the links on the website now have proper .html extensions, which I had to remove previously to get the gzip negotation to work.

(If you still see something newly broken, drop me a note by comment or email, of course. Then again, if you're reading this, you obviously are seeing something, which is a lot more than a few hours ago.)

I use compression mainly out of habit, from the days when I had a lot less bandwidth -- around 50-200MB/day -- and ran into bandwidth issues even with mainly just text pages. This was around the time when Netscape 4 and IE4 were taking off and thus gzip compression support in browsers was becoming widely available, and 2-3:1 compression was a huge deal. Nowadays, I have a ton more bandwidth than I'm using, but I still like to strip whitespace and use gzip because I hate unnecessary waste and it loads faster over slow links. That probably means I could stand to use a lot more images, but I guess what I have now better suits my total lack of web page aesthetic sense.

Web page design and administration isn't really my specialty, as you might have guessed. Even though I don't have a lot of spare time and what I do have tends to be applied toward coding, I still find it useful to maintain the website and especially the blog. The blog's reputation for being a spew fest for random thoughts is well-deserved -- but what I find amazing is the response that I get despite my own peculiar mix of topics. The amazing, and slightly scary, aspect of the Internet is that there are so many people connected that there's nearly always at least one person interested in what you are saying, no matter what it is.

(Read more....)

§ ¶Why doesn't VirtualDub enable the new Windows XP visual style?

Sporadically, I get people emailing me .manifest files to enable the Windows XP look on buttons and sliders in VirtualDub's UI and ask me why I don't ship with them enabled.

Because it would break plugins and codecs.

Microsoft realized that it would be difficult to redo the look of the common controls without introducing compatibility issues, so in Windows XP they introduced side-by-side versions of the libraries. Version 5 of COMCTL32.DLL still sports the old look and is used by default; applications can opt-in to COMCTL32.DLL Version 6. The problem is that the most straightforward way of doing this is to use an application manifest, which in VirtualDub's case happily causes all DLL loaded in the process, including plugins and video codecs, to use the new visual style too... which could then break their configuration dialogs. If you think this is theoretical, it isn't. When I tried it, it took me about five seconds to spot the first problem in the main app alone: the mark-in/mark-out buttons don't display properly with V6.

Now, Microsoft's solution for this is to compile an application as "Isolation Aware," meaning that the application uses the common controls library in such a way that only it, and other DLLs can still use V5. In theory, enabling this would allow VirtualDub to use visual styles without affecting plugins. Yay! Problem is, it isn't clear how this feature affects down-level compatibility with Windows 98/ME/2000, and it it's implemented as a bunch of inlines in the header that have code like this:

    const BOOL fActivateActCtxSuccess = IsolationAwarePrivateT_SqbjaYRiRY || IsolationAwarePrivatenPgViNgRzlnPgpgk(&ulpCookie);

Just the thing I want to enable blindly in my app. Not. Thanks for obfuscating this, guys. I haven't had time to decipher this and look at all of the ramifications of enabling this, or to search for instances of dynamic linking with GetProcAddress() (which wouldn't work in this kind of scheme).

I think that in worst case I can use the activation context API to conditionally enable V6 and add an option to Preferences to toggle it, but to do that I'd definitely have to make all calls into COMCTL32 dynamic. Bleah.

(Read more....)

§ ¶VirtualDub 1.7.3 released

Timeout expired... write buffers flushed. New version released.

1.7.3 is out. This is again an experimental release, but it mostly contains bug fixes. It's been too long since the last release and some bug fixes that were too risky or extensive for 1.6.x had been piling up, so I decided to push it out. I've begun shunting new experimental features to a secondary branch and so the plan is to try to bake this code line until we can go stable again -- no new features. Please report any problems, and I'll do more experimental releases as necessary until the code line stabilizes.

The display code has been solidified a bit again and so a bunch of the refresh problems introduced from the multithreading experiments and the Vista DWM related tweaks should be gone. During the past couple months I'd been exercising the display code heavily in some other apps, which helped expose and wring out a number of bugs.

Notable new feature: I had gotten tired of constantly hacking my local build in order to play PS2 games on my laptop with low latency, so I implemented some DirectShow clocking options in the capture module. Disabling the clock is the one that helps turn your computer into an expensive television set, but there is now also an option to use audio timestamps. Previously VirtualDub always estimated audio timestamps from the CPU clock and the video clock, which was necessary under the VFW model; this was, however, not always very stable. The timestamps on the DirectShow audio samples can be considerably more accurate due to possibly being provided by the kernel driver. If you do video capture, go to Options > Timing and give it a whirl; it may make capturing more stable.

Changelog after the jump.

(Read more....)