Current version

v1.10.4 (stable)

Navigation

Main page
Archived news
Downloads
Documentation
   Capture
   Compiling
   Processing
   Crashes
Features
Filters
Plugin SDK
Knowledge base
Contact info
Forum
 
Other projects
   Altirra

Search

Archives

01 Dec - 31 Dec 2013
01 Oct - 31 Oct 2013
01 Aug - 31 Aug 2013
01 May - 31 May 2013
01 Mar - 31 Mar 2013
01 Feb - 29 Feb 2013
01 Dec - 31 Dec 2012
01 Nov - 30 Nov 2012
01 Oct - 31 Oct 2012
01 Sep - 30 Sep 2012
01 Aug - 31 Aug 2012
01 June - 30 June 2012
01 May - 31 May 2012
01 Apr - 30 Apr 2012
01 Dec - 31 Dec 2011
01 Nov - 30 Nov 2011
01 Oct - 31 Oct 2011
01 Sep - 30 Sep 2011
01 Aug - 31 Aug 2011
01 Jul - 31 Jul 2011
01 June - 30 June 2011
01 May - 31 May 2011
01 Apr - 30 Apr 2011
01 Mar - 31 Mar 2011
01 Feb - 29 Feb 2011
01 Jan - 31 Jan 2011
01 Dec - 31 Dec 2010
01 Nov - 30 Nov 2010
01 Oct - 31 Oct 2010
01 Sep - 30 Sep 2010
01 Aug - 31 Aug 2010
01 Jul - 31 Jul 2010
01 June - 30 June 2010
01 May - 31 May 2010
01 Apr - 30 Apr 2010
01 Mar - 31 Mar 2010
01 Feb - 29 Feb 2010
01 Jan - 31 Jan 2010
01 Dec - 31 Dec 2009
01 Nov - 30 Nov 2009
01 Oct - 31 Oct 2009
01 Sep - 30 Sep 2009
01 Aug - 31 Aug 2009
01 Jul - 31 Jul 2009
01 June - 30 June 2009
01 May - 31 May 2009
01 Apr - 30 Apr 2009
01 Mar - 31 Mar 2009
01 Feb - 29 Feb 2009
01 Jan - 31 Jan 2009
01 Dec - 31 Dec 2008
01 Nov - 30 Nov 2008
01 Oct - 31 Oct 2008
01 Sep - 30 Sep 2008
01 Aug - 31 Aug 2008
01 Jul - 31 Jul 2008
01 June - 30 June 2008
01 May - 31 May 2008
01 Apr - 30 Apr 2008
01 Mar - 31 Mar 2008
01 Feb - 29 Feb 2008
01 Jan - 31 Jan 2008
01 Dec - 31 Dec 2007
01 Nov - 30 Nov 2007
01 Oct - 31 Oct 2007
01 Sep - 30 Sep 2007
01 Aug - 31 Aug 2007
01 Jul - 31 Jul 2007
01 June - 30 June 2007
01 May - 31 May 2007
01 Apr - 30 Apr 2007
01 Mar - 31 Mar 2007
01 Feb - 29 Feb 2007
01 Jan - 31 Jan 2007
01 Dec - 31 Dec 2006
01 Nov - 30 Nov 2006
01 Oct - 31 Oct 2006
01 Sep - 30 Sep 2006
01 Aug - 31 Aug 2006
01 Jul - 31 Jul 2006
01 June - 30 June 2006
01 May - 31 May 2006
01 Apr - 30 Apr 2006
01 Mar - 31 Mar 2006
01 Feb - 29 Feb 2006
01 Jan - 31 Jan 2006
01 Dec - 31 Dec 2005
01 Nov - 30 Nov 2005
01 Oct - 31 Oct 2005
01 Sep - 30 Sep 2005
01 Aug - 31 Aug 2005
01 Jul - 31 Jul 2005
01 June - 30 June 2005
01 May - 31 May 2005
01 Apr - 30 Apr 2005
01 Mar - 31 Mar 2005
01 Feb - 29 Feb 2005
01 Jan - 31 Jan 2005
01 Dec - 31 Dec 2004
01 Nov - 30 Nov 2004
01 Oct - 31 Oct 2004
01 Sep - 30 Sep 2004
01 Aug - 31 Aug 2004

Stuff

Powered by Pivot  
XML: RSS feed 
XML: Atom feed 

§ Appending streams and mismatch errors

I see that one of the append dialogs from VirtualDub has been featured in The Daily WTF. Personally, I consider the presentation of this dialog as a WTF to be a grave disservice, since there are so many better WTFs in the program that could have been used. I feel insulted. I suppose I should explain the reasoning behind this dialog, though, and why it pops up annoyingly when it does.

To clear up a misconception: one of the comments noted that the error was caused by comparison in floating-point. This actually isn't the case, and if it were, it would actually make the dialog appear less frequently due to roundoff. (Remember, floating-point can be inaccurate, but consistently so.) Frame rates in AVI are not stored as floating-point or fixed-point, but in rational form as the ratio of two 32-bit unsigned numbers. This means that the sample rates of two streams can differ by up to the 10th significant digit, and more importantly, two streams can have different values in the header that correspond to the exact same frame rate. The submitter didn't indicate what version he was using, but versions prior to 1.4 have a bug in that they don't do a proper fraction check (they check numerator and denominator independently); this can cause error messages like the submitted, where the two frame rates are the same. One way to cause this is to process one of the segments with a version of VirtualDub that normalizes the frame rate fraction to lowest common denominator  the starting version of which I forget  so make sure you're using the newer versions of VirtualDub across the board when possible, to use the more liberal check.

And, yes, admittedly the error is not very informational, and I should look into clarifying it. However, it exists because of a sticky limitation with the way append is implemented, and I won't apologize for making an overly technical error message instead of one that says "I can't append."

What the append command is actually for

VirtualDub isn't a non-linear editor, and has never been one. In fact, neither the rendering engine nor the UI can handle more than one input video and audio stream. This isn't to say I don't want it to be one, but to get that far takes a lot of work, and there's a lot that I do or want to do with the program that doesn't require NLE support and is a lot easier to implement. I think I could buy software acceptable to my future NLE needs more easily than I could obtain one for my peculiar video capture requirements.

What is the append function for, then? It's to re-splice a movie that has been split across multiple files to circumvent file size limits, mainly from video capture. Due to the granularity of audio and video streams, particularly when audio compression is in use, the files can't be cut exactly cleanly, and thus the video stream will be slightly longer than the audio stream, or vice versa. For this reason the Append function does an unaligned splice. This means that the audio and video streams are independently glued end-to-end without regard for sync to each other, so any difference in the lengths of the streams in the current file will correspondingly shift sync in the appended segment. This behavior is actually desired for a split movie, because it undoes the desync that occurred when the split occurred. It's a little less desirable when you're trying to glue independent videos together.

So using this function to splice together a bunch of different clips to make a montage isn't exactly the original intended use, although admittedly it's the closest you'll find in the program.

What this means is that if you're trying to use the feature to join two different video files, you have to be a bit careful about how the first file is trimmed. You want the durations of the audio and video to match as closely as possible to minimize the desync on the second segment. One way to do this is to purposely cut a few frames off the end of the first segment and reprocess it in direct/direct mode; unless the "cut off" setting has been disabled in Video > Select Range, VirtualDub will trim the audio as closely as possible to the length of the video, assuming you cut back far enough. The finer the granularity of the audio, the better, so uncompressed audio will give you a more precise cut here.

Where the restrictions come from

I mentioned earlier that the main subsystems can't handle more than one A/V stream each. So where is the join handled? Well, it's actually done in the AVI parser by merging the indices and virtually concatenating the raw files, which is the reason why you can't append anything but AVIs, and is also the reason why you can't append Avisynth scripts because the AVI layer isn't parsing a file in that path. The only time you can append is between two real AVI files that VirtualDub is itself parsing. I forget why I put it in the parser, but I think it's because it was a lot easier than trying to manage a lot of individual parsers and trying to consolidate the buffer caches between them (memory was a bit tighter then).

As for the other restrictions....

The exact restrictions in the current version, 1.6.11, are:

The data format check trips up some people when they attempt to recompress one video to match the other. The data formats in the two streams must match exactly. VirtualDub enforces this because it has no idea what the data format block contains, since most of it is opaque and specific to each video/audio codec, and decoding a stream with the wrong format can easily cause a crash. For this reason, I do not intend to make this particular check optional. Note that there is a bug in Huffyuv 2.1.1 that can trip this check unnecessarily, since it fails to initialize a few bytes at the end of its format structure. VirtualDub pre-clears the format block memory to avoid this problem on write, but until recently this was not done in capture mode until 1.6.9+, so if you capture a Huffyuv file with 1.6.8 or earlier and attempt to merge it with another Huffyuv file, it may fail for this reason.

The frame rate check is, of course, the one featured in the WTF dialog. The first issue is that people attempt to look at the number in the error and type it in to the frame rate adjustment box to coerce to that rate, but that doesn't work, because the single fractional number you can enter for the frame rate isn't enough to specify the fraction needed to match, and there isn't UI for entering in the fraction directly. (If anyone has a good algorithm to compute the closest 32-bit/32-bit fraction given a single real number, please contact me.) The second issue is that any attempt to coerce the second stream to the frame rate of the first will change the speed of the second stream. This is the real reason behind the frame rate error. Changing the length of the video stream has two bad effects: it causes a gradual resync relative to the audio stream, and it also splits the stream ends apart in time, which then causes more desync on any subsequent appends. Neither of these effects are readily apparent in the UI. Also, very small errors become magnified after 3600+ seconds of video.

Now, the question I'm sure you're asking is: why doesn't the parser just ask whether the error is acceptable or not? Well, the AVI parser code is too low-level to display UI, and while it can queue up warnings for display later, that doesn't work for user queries. One reason for this restriction is to avoid reentrancy problems caused by running message loops in non-UI code. (This is, by the way, why it is a bad idea to spawn message boxes in the main thread from a video codec message that isn't UI related.) So, doing this is a bit inadvisable. Also, I simply don't like this workaround.

MP3 audio also can cause sample rate match errors on the audio stream, because of a quirk in the Fraunhofer-IIS MP3 codec namely that it writes an inaccurate sample rate for 44KHz MP3 streams. Or rather, it writes the correct rate, but doesn't target it exactly when encoding, often off by ~0.5%, which is enough to cause perceptible desynchronization. VirtualDub recomputes the closest value when using an audio codec that encodes to MP3, but the correct value is frequently fractional and thus can differ by +/-1, and furthermore, its calculation might not match exactly with another program that does the same correction. This correction is also disabled during capture and when writing a segmented AVI. Therefore, it is recommended that this particular configuration be avoided when using the append feature.

TODO

What people are wanting when they run into problems with Append is an aligned splice, which can edit the streams until they can be spliced without holes in either stream. To do this the append needs to happen at a higher layer than the parser, probably in the decodable stream layer. This is probably not too bad to do for streams that do match in frame rate, but doing it for those that don't is harder because I actually need to resample one of the streams, probably the video stream. And that's complicated by the fact that in the general case, I can insert frames, but not delete them. So there's a bit of legwork to be done here to figure out how to do this smoothly, and I haven't had time to do this yet.

Comments

Comments posted:


so thats why i couldnt join those files that "appeared" to match so perfectly having the same frame size, frame rate, video codec and audio sample rate.
it was just not made to do that particular task.
very clarifying, thanks.

unaligned - 15 10 05 - 05:57


For your aligned splice, if you can't remove any frames, how about taking the highest fps and create duplicated frames or merging frames to create new frames to match to the fps? This feature could be interesting because the unaligned splice is quite restrictive.

Also, maybe this website could help you for your 32-bit/32-bit fraction from a decimal (http://mathforum.org/library/drmath/view..).

Hope this would help you.

Simbou - 15 10 05 - 19:04


"decimal" (or floating point) to fraction:

http://mathforum.org/dr.math/faq/faq.fra..

this is also the algorithm that matlab uses (function "rat"), so it gotta be good...

anon - 15 10 05 - 22:42


re: floating -> fraction: even better, an example of how it's done:

http://en.wikipedia.org/wiki/Continued_f..

anon - 15 10 05 - 23:02


If I understand the request right, he/she wants a good algorithm to do it. While the mathforum way is technically correct, I don't think it fits well with the "good" requirement.
Though I don't quite understand why Avery wants a good algorithm for something that is only used at one place... or is there some other place where he/she needs it to be good?

Marcellus - 16 10 05 - 08:32


Maybe it's just me, but wouldn't a simple "Advanced" button next to the "Change to ... frames per second" field in the frame rate menu be enough? It could just bring up a small dialogue box where the user can set dwRate and dwScale by himself.

porblem solved lol

Yuri - 16 10 05 - 12:26


By "good algorithm," I mean one that executes in reasonable time. Counting denominators from 1 to 0xFFFFFFFF and choosing the fraction with the least error takes forever and is not a good algorithm. Continued fractions, however, look rather good in fact, it looks like they'd produce a result in logarithmic time. (Thanks for the link!) Unfortunately, there is also the problem that any frame rate in decimal form that has less than 10 significant digits has a trivial result in this case, so there are some heuristics required. What I could also use this for, though, is to get a more precise result when the fraction result overflows (right now it just forces a denominator of 0xFFFFFFFF).

Re: Advanced rate/scale dialog. I might have to do this, slight issue being that you can't actually see the rate/scale fraction of a source anywhere but the hex editor, so I'd have to add that too. Otherwise you might as well just set the frame rate in the hex editor as well, since it'd be faster than converting to decimal and copying it over.

Re: Adding/removing frames. This basically amounts to resampling the video stream. Removing or merging (interpolating) wouldn't really work unless I put it in the render engine, which would be a lot of work, but adding might work. Would introduce up to a half a frame of error at some point in the video, but that's unavoidable without interpolation. Also wouldn't help with audio, but that's not usually as much of a problem. Hmmm.

Phaeron - 18 10 05 - 01:27


As for UI to force override splice restictions, just a check box on the append file browser.

From the avisynth core to go from a 32bit float to rational fraction :-

double n = args[1].AsFloat();
unsigned d = 1;
while (n < 16777216 && d < 16777216) { n*=2; d*=2; } // 2^24, floats precision
unsigned num=unsigned(n+0.5)
and then normalize if desired.

You could also directly bit strip the floats mantisa and exponent.

IanB

IanB - 18 10 05 - 05:20


It would be nice to have a command in the Edit menu that would cut off video/audio so the ends match as closely as possible without losing too much data.

BiShop - 18 10 05 - 10:12


Bishop: from what I understood, you can't easily cut audio to a very precise timestamp due to lack of granularity: compressed audio isn't very granular (mp3 for example, due to the bit reservoir, isn't so good), so it can't be very precisely cut at the parser level - that's why it would need to be processed, THEN cut. For that to work and not having to reprocess the whole stream, Vdub would need to be able to make a direct copy up to the point where it encounters a keyframe, then process the following part (both sound and video), do the evoked resampling work for perfect joining, then resume its direct copying - but this, if I'm not mistaken, falls more into the area of non-linear edition, so if you want to go that far, get a copy of Adobe Premiere. I hope I understood Avery well.

Mitch74 - 19 10 05 - 03:47


Is anybody working to [fix] this? If I can't take excerpts from a video recording and stitch them together, I still love VirtualDub for trimming (cutting out sections of video), but I can't promote it - I thought I had found the holy grail, then found this limitation. Or now I realize maybe it's not such a problem - instead of exporting excerpts and then trying to append them together, I can just trim within the main file. Ah, but this requires that I capture all the videotape in one take, one massive file - if I want 1 minute from the beginning and one minute from the end, I need to capture the whole tape. Unless ... can we append clips if they are in [almost uncompressed] audio and video?. (Did Avery imply that?) That requires we have free space, but seems easier than trying to manually make the files compatible for appending.

Jonathon (link) - 03 01 07 - 03:37

Comment form


Please keep comments on-topic for this entry. If you have unrelated comments about VirtualDub, the forum is a better place to post them.
Name:  
Remember personal info?

Email (Optional):
Your email address is only revealed to the blog owner and is not shown to the public.
URL (Optional):
Comment: /

An authentication dialog may appear when you click Post Comment. Simply type in "post" as the user and "now" as the password. I have had to do this to stop automated comment spam.



Small print: All html tags except <b> and <i> will be removed from your comment. You can make links by just typing the url or mail-address.