§ ¶Is it too much to ask to have ONE good image display API in Windows?
Lately, I've been becoming increasingly frustrated with how difficult it is in Windows to reliably and efficiently blit an image to the screen with high quality. It shouldn't be that hard, but it is, because there are half a dozen different ways to do so and none of them meet all of the requirements. So I sat down and made a table of all of the ways to blit an image to the screen in Windows, and how they all suck in some fashion.
VirtualDub has code paths for GDI, DirectDraw blit, DirectDraw overlay, Direct3D, and OpenGL. GDI+ is here because it looks like a good API, until you discover that it has no useful hardware acceleration, has incorrect subpixel positioning for image blits, and is no longer being evolved. I put WPF (Avalon) here because I looked into it as a possible alternative when operating under DWM composition / Aero Glass on Vista, which is problematic since neither GDI nor DirectDraw are accelerated, and Direct3D in child windows seems very flaky. The huge problem with WPF is that it requires .NET managed code, since the API is in .NET and the underlying MIL API isn't documented (grumble); another problem is that it seems unusually slow and flickers a lot whenever windows are resized.
Anyway, the table of image blitting woe:
|
GDI |
GDI+ |
DirectDraw
(blit) |
DirectDraw
(overlay) |
Direct3D |
OpenGL |
WPF (Avalon) |
| Platform support |
95+
NT3.1+ |
98+
NT4+[1] |
98+
NT4+[2] |
98+
NT4+[2] |
98+
NT4+[2] |
driver |
XP+[12] |
| Requires managed code |
no |
no |
no |
no |
no |
no |
yes |
| Hardware accel w/o 3D HW |
yes |
no |
yes |
yes |
no |
no |
no |
| Hardware accel with 3D HW |
yes |
no |
yes |
yes |
yes |
yes |
yes |
| Software fallback |
yes |
yes |
yes |
no |
yes [3] |
yes [4] |
yes [3] |
| Works with DWM composition |
sw |
sw |
sw |
no [5] |
yes |
yes |
yes |
| Bilinear filtering |
sw [13] |
sw |
yes [6] |
yes [7] |
yes |
yes |
yes |
| Bicubic filtering |
no |
sw |
no |
no |
yes [8] |
yes [8] |
sw |
| Terminal Services |
sw |
sw |
sw |
no |
no |
no |
sw [9] |
| Supports 256 color display |
yes |
yes |
yes |
yes |
no |
no |
? |
| RGB format conversion |
yes |
sw |
no |
no [10] |
yes |
yes |
yes |
| YCbCr format conversion |
no |
no |
no |
yes |
yes |
yes |
no |
| Beam detection |
no |
no |
yes |
yes |
yes |
no |
no |
| Beam avoidance (vsync) |
no [11] |
no [11] |
yes |
yes |
yes |
yes |
no [11] |
Explanations:
- sw: Supported, but with software emulation only.
- Platform support: Versions of Windows on which this API is available, including ones for which a redistributable is required.
- Requires managed code: Whether this API can only be used from .NET managed code.
- Hardware acceleration w/o 3D HW: Whether this API can be accelerated with older graphics hardware that only supports 2D acceleration.
- Hardware acceleration with 3D HW: Whether this API can be accelerated on 3D-capable hardware.
- Software fallback: If operation is possible without hardware support.
- Works with DWM composition: Operation when DWM composition (Aero Glass) is active under Windows Vista.
- Bilinear filtering: If bilinear filtering (4 tap) is supported on stretched images.
- Bicubic filtering: If bicubic filtering (16 tap) is supported on stretched images.
- Terminal Services: Whether the API works over Terminal Services (Remote Desktop) is active.
- Supports 256 color display: If operation is possible on a paletted display.
- RGB color conversion: If display of an RGB image in a different format than the display buffer is supported.
- YCbCr color conversion: If display of a YCbCr-encoded image is supported.
- Beam detection: If the API supports reading the position of the display image scanning beam.
- Beam avoidance (vsync): If the API supports altering the timing of image display to avoid tearing.
Notes:
- Requires redistributable prior to Windows XP.
- Requires redistributable for Windows 95.
- With RGBRast. (Refrast is not counted as it requires the SDK and is excruciatingly slow.)
- Microsoft's OpenGL 1.1 software implementation is available, but it is very slow.
- Not supported. Overlay creation succeeds, but the overlay never shows up.
- DirectDraw blits are point-sampled when DWM composition (Aero Glass) is active. Otherwise, filtering is up to the driver.
- Varies widely; some drivers don't interpolate vertically, and some only interpolate chroma.
- Requires custom implementation.
- Can be hardware accelerated between two Vista-based systems using Avalon Remoting.
- RGB overlays are possible, but I've never seen hardware that supported it.
- Automatic if DWM composition (Aero Glass) is enabled.
- Requires redistributable.
- Requires Windows NT; quite slow.
Comments
Comments posted:
So, what are you gonna do?
Carter - 05 03 07 - 06:09
Looking at your table, you should consider OpenGL...
Here, it also works fine across Remote Desktop and the fact that you can use Pixel Shaders for implementing realtime video effects is also a big advantage.
Martin - 05 03 07 - 07:51
I like OpenGL myself, I think the downsides listed here are limited to fairly unusual circumstances. How many computers these days
don't have 3d hardware? The other advantage with OpenGL is that it will allow the program to run more easily in wine under Linux.
TechMage89 - 05 03 07 - 16:30
I thought Wine handled DDraw and the rest of DirectX up to about 7 pretty well at this point? I suppose that depends on how new your Wine is.
Entirely off topic, I noticed that the table was the same color scheme as the old site was, before it got the hip update. ;p
foxyshadis - 05 03 07 - 22:31
What I've been doing is just supporting several code paths. For Vista, though, I still don't have a good solution. I've been too lazy to install Vista on my desktop to determine if the Direct3D problems I'm seeing are Microsoft's fault or NVIDIA's fault (my desktop has an ATI card).
OpenGL is not viable over Remote Desktop -- at that point you're using Microsoft's OpenGL 1.1 software implementation, which is quite slow. It's an order of magnitude slower than a JIT-ing 3D software rasterizer like Pixomatic, and even slower than just a custom blitter going through DirectDraw. You also have no fragment shader support, just plain texture environment blending. The best API for Remote Desktop / Terminal Services would be Avalon, if the damn thing weren't restricted to managed code. Hosting the CLR is a PITA. :-/
All of the pixel/fragment shaders I use are sub-PS2.0 profile, so I doubt Wine would have a problem even with my Direct3D code. Last time I checked, though, its DirectDraw clipping code was
still broken.
As for the color scheme, well, that's just me picking pastel RGB colors at random that aren't reddish....
Phaeron - 05 03 07 - 23:48
I played with WPF recently and bumped into that undocumented (and with reflector a really scary looking :) layer too. The only way to get pixels from memory on screen is the WritableBitmap class, but even when updating a 1x1 rectange it throws away the whole image and allocates a new surface inside. WPF is nice but not any useful for video.
Gabest - 06 03 07 - 01:17
Cycling textures doesn't worry me that much, because that usually happens anyway in order to avoid stalling on the 3D device -- if you were bypassing managed textures in Direct3D and going straight to default pool, you would want to cycle between at least 2-3 device textures as well to avoid pipeline stalls. What worries me a lot more is the flickery mess just putting an image in a window. There's absolutely no reason for that to happen in a retained mode 2D system.
The other thing to keep in mind, however, is that the underlying MIL API is so called because it is the
Media
Integration
Layer. It is supposed to have hookups for integrating video as well as 2D drawing primitives. Clearly, there is some way within MIL to integrate video -- the question is whether it's any different than just using plain old Direct3D. WPF itself, though, still seems unpolished and underoptimized. I've heard that 3D models displayed through WPF are actually lit in software instead of using the vertex shader.
If you're really bored, look at the resources within MILCORE.DLL -- some of them are binary vertex and pixel shaders, some are compiled effects, and two of them are actually .fx files (!). Whoever worked on the .fx files didn't have much HLSL experience and computed "sqrt(dx*dx + dy*dy)" instead of just using "length(v)"; I suspect that the radial gradient could also have been done in ps1.1 or at least ps1.4, with appropriate wilyness (normalization cube map or such).
Phaeron - 06 03 07 - 02:33
If you only want to work with still images then it's not that bad, but seeing how the memory usage in taskman jumps between 50 and 150 megs while the GC is trying hard to recycle those bitmaps then you start worring about it. There is also serious tearing, but it could be because this integrated intel video card I have here for vista doesn't have a new kind of driver.
Gabest - 06 03 07 - 19:17
Comment form
Please keep comments on-topic for this entry. If you have unrelated comments about VirtualDub, the forum is a better place to post them.
