Current version

v1.10.4 (stable)

Navigation

Main page
Archived news
Downloads
Documentation
   Capture
   Compiling
   Processing
   Crashes
Features
Filters
Plugin SDK
Knowledge base
Donate
Contact info
Forum
 
Other projects
   Altirra

Search

Archives

01 Dec - 31 Dec 2013
01 Oct - 31 Oct 2013
01 Aug - 31 Aug 2013
01 May - 31 May 2013
01 Mar - 31 Mar 2013
01 Feb - 29 Feb 2013
01 Dec - 31 Dec 2012
01 Nov - 30 Nov 2012
01 Oct - 31 Oct 2012
01 Sep - 30 Sep 2012
01 Aug - 31 Aug 2012
01 June - 30 June 2012
01 May - 31 May 2012
01 Apr - 30 Apr 2012
01 Dec - 31 Dec 2011
01 Nov - 30 Nov 2011
01 Oct - 31 Oct 2011
01 Sep - 30 Sep 2011
01 Aug - 31 Aug 2011
01 Jul - 31 Jul 2011
01 June - 30 June 2011
01 May - 31 May 2011
01 Apr - 30 Apr 2011
01 Mar - 31 Mar 2011
01 Feb - 29 Feb 2011
01 Jan - 31 Jan 2011
01 Dec - 31 Dec 2010
01 Nov - 30 Nov 2010
01 Oct - 31 Oct 2010
01 Sep - 30 Sep 2010
01 Aug - 31 Aug 2010
01 Jul - 31 Jul 2010
01 June - 30 June 2010
01 May - 31 May 2010
01 Apr - 30 Apr 2010
01 Mar - 31 Mar 2010
01 Feb - 29 Feb 2010
01 Jan - 31 Jan 2010
01 Dec - 31 Dec 2009
01 Nov - 30 Nov 2009
01 Oct - 31 Oct 2009
01 Sep - 30 Sep 2009
01 Aug - 31 Aug 2009
01 Jul - 31 Jul 2009
01 June - 30 June 2009
01 May - 31 May 2009
01 Apr - 30 Apr 2009
01 Mar - 31 Mar 2009
01 Feb - 29 Feb 2009
01 Jan - 31 Jan 2009
01 Dec - 31 Dec 2008
01 Nov - 30 Nov 2008
01 Oct - 31 Oct 2008
01 Sep - 30 Sep 2008
01 Aug - 31 Aug 2008
01 Jul - 31 Jul 2008
01 June - 30 June 2008
01 May - 31 May 2008
01 Apr - 30 Apr 2008
01 Mar - 31 Mar 2008
01 Feb - 29 Feb 2008
01 Jan - 31 Jan 2008
01 Dec - 31 Dec 2007
01 Nov - 30 Nov 2007
01 Oct - 31 Oct 2007
01 Sep - 30 Sep 2007
01 Aug - 31 Aug 2007
01 Jul - 31 Jul 2007
01 June - 30 June 2007
01 May - 31 May 2007
01 Apr - 30 Apr 2007
01 Mar - 31 Mar 2007
01 Feb - 29 Feb 2007
01 Jan - 31 Jan 2007
01 Dec - 31 Dec 2006
01 Nov - 30 Nov 2006
01 Oct - 31 Oct 2006
01 Sep - 30 Sep 2006
01 Aug - 31 Aug 2006
01 Jul - 31 Jul 2006
01 June - 30 June 2006
01 May - 31 May 2006
01 Apr - 30 Apr 2006
01 Mar - 31 Mar 2006
01 Feb - 29 Feb 2006
01 Jan - 31 Jan 2006
01 Dec - 31 Dec 2005
01 Nov - 30 Nov 2005
01 Oct - 31 Oct 2005
01 Sep - 30 Sep 2005
01 Aug - 31 Aug 2005
01 Jul - 31 Jul 2005
01 June - 30 June 2005
01 May - 31 May 2005
01 Apr - 30 Apr 2005
01 Mar - 31 Mar 2005
01 Feb - 29 Feb 2005
01 Jan - 31 Jan 2005
01 Dec - 31 Dec 2004
01 Nov - 30 Nov 2004
01 Oct - 31 Oct 2004
01 Sep - 30 Sep 2004
01 Aug - 31 Aug 2004

Stuff

Powered by Pivot  
XML: RSS feed 
XML: Atom feed 

§ How to cause a stack overflow in Notepad using DirectShow filters

To cause a stack overflow in Windows XP Notepad:

Boom.

This crash occurs because the standard Windows Explorer-based file dialog tries to use DirectShow to determine video file specific information for the video file using DirectShow, and because of the lame way in which some DirectShow filters force themselves into the filter graph created by Windows Media Player, which as a side effect can screw up other programs that use DirectShow. If you install DirectShow filters that have such an "auto-enable on playback" option, please leave it off when using other audio/video programs that may use DirectShow, such as video capture applications.

Why this happens:

DirectShow constructs video filter graphs using an "intelligent" mechanism that attempts to find filters that will get a graph to a desired result. The Registry contains a list of video filters, along with their class (filter, compressor, capture source, etc.) and priority. The filter graph manager keeps adding the highest compatible filters to a graph until it finally has an output that a renderer can accept, finishing the graph.

Say you're trying to play a video file, for instance.

Well, it's an AVI file, so a File Reader and an AVI Splitter are inserted. The AVI Splitter outputs compressed video, which the Video Renderer can't handle, but both the AVI Decompressor and MJPEG Decompressor can. The MJPEG Decompressor has higher priority, so it gets inserted, and it now outputs RGB video that the Video Renderer can handle. And we're done.

The intelligent connect/render mechanism has the advantage that filters can be added to the Registry and they will automatically enable new connection paths that weren't possible before. The disadvantage is that some of the paths that the filter graph manager creates are a bit... suboptimal. Like using a Motion JPEG compressor followed by a Motion JPEG decompressor to convert between two different RGB formats.

How does this pertain to stack overflows? Well, the way that some filters insert themselves into the Windows Media Player filter graph is to advertise themselves as a renderer at maximum priority. This ensures that they are always inserted first into the graph, but then they still have an output pin that needs to be rendered. The filter graph manager then tries to insert another instance of that filter, but its query function returns false this time because the filter refuses to connect to an instance of the same class. The filter graph manager then tries the next renderer filter, which succeeds. The problem is that if two filters are installed that try to do this, the filter graph manager keeps inserting both of them into the graph alternatingly in an ever-increasing chain until the thread dies with a stack overflow due to excessive call depth.

In the case of VirtualDub, the two places you are likely to encounter this problem is either trying to open a video file or selecting a capture device in capture mode. Stack overflows are really nasty in that there is only ~4K of stack to use when the exception is thrown; VirtualDub's crash handler attempts to switch stacks ASAP by creating a new thread, and if you are lucky, the call stack in the Advanced view will show which DirectShow filters are responsible. If you're not, though, the application will simply vanish and you will have to guess. Usually it is undesirable for DSP filters to be active during a video capture, so it's best to disable all of them before capturing video.

Comments

Comments posted:


Wow... Windows is really... intelligent

TechMage89 - 20 04 06 - 16:57


"Stack overflows are really nasty in that there is only ~4K of stack to use when the exception is thrown"
Newer version of Windows can increase the limit by using SetThreadStackGuarantee.

Yuhong Bao - 27 07 09 - 17:28

Comment form


Please keep comments on-topic for this entry. If you have unrelated comments about VirtualDub, the forum is a better place to post them.
Name:  
Remember personal info?

Email (Optional):
Your email address is only revealed to the blog owner and is not shown to the public.
URL (Optional):
Comment: /

An authentication dialog may appear when you click Post Comment. Simply type in "post" as the user and "now" as the password. I have had to do this to stop automated comment spam.



Small print: All html tags except <b> and <i> will be removed from your comment. You can make links by just typing the url or mail-address.