Wikipedia

DirectShow

(Redirected from Video Mixing Renderer)

DirectShow (sometimes abbreviated as DS or DShow), codename Quartz, is a multimedia framework and API produced by Microsoft for software developers to perform various operations with media files or streams. It is the replacement for Microsoft's earlier Video for Windows technology.[1] Based on the Microsoft Windows Component Object Model (COM) framework, DirectShow provides a common interface for media across various programming languages, and is an extensible, filter-based framework that can render or record media files on demand at the request of the user or developer. The DirectShow development tools and documentation were originally distributed as part of the DirectX SDK.[2] Currently, they are distributed as part of the Windows SDK (formerly known as the Platform SDK).[3]

DirectShow
Other namesDShow
Developer(s)Microsoft
Operating systemMicrosoft Windows
PredecessorActiveMovie
TypeMultimedia framework and API
LicenseProprietary commercial software
Websitedocs.microsoft.com/en-us/windows/win32/directshow/directshow

Microsoft plans to completely replace DirectShow gradually with Media Foundation in future Windows versions. One reason cited by Microsoft is to provide "much more robust support for content protection systems"[4] (see digital rights management). Microsoft's Becky Weiss confirmed in 2006 that "you'll notice that working with the Media Foundation requires you to work at a slightly lower level than working with DirectShow would have. And there are still DirectShow features that aren't (yet) in Media Foundation".[4][5] As described in the Media Foundation article, Windows Vista and Windows 7 applications use Media Foundation instead of DirectShow for several media related tasks.

History

edit

The direct predecessor of DirectShow, ActiveMovie (codenamed Quartz), was designed to provide MPEG-1 support for Windows. It was also intended as a future replacement for media processing frameworks like Video for Windows and the Media Control Interface, which had never been fully ported to a 32-bit environment and did not utilize COM.[6][1]

The development team used a pre-existing modular digital-media-processor project codenamed "Clockwork" as a basis for DirectShow. Clockwork had previously been used in the Microsoft Interactive Television project.[7]

The project was initially named "ActiveMovie", and was released in May 1996, bundled with the beta version of Internet Explorer 3.0.[8][9] In March 1997, Microsoft announced that ActiveMovie would become part of the DirectX 5 suite of technologies, and around July started referring to it as DirectShow, reflecting Microsoft's efforts at the time to consolidate technologies that worked directly with hardware under a common naming scheme.[10][11][12] DirectShow became a standard component of all Windows operating systems starting with Windows 98;[13] however it is available on Windows 95 by installing the latest available DirectX redistributable.[14] In DirectX version 8.0, DirectShow became part of the core DirectX SDK along with other DirectX APIs.[15]

In October 2004, DirectShow was removed from the main DirectX distribution and relocated to the DirectX Extras download.[citation needed] In April 2005, DirectShow was removed entirely from DirectX and moved to the Windows SDK starting with the Windows Server 2003 SP1 version of the SDK.[3] The DirectX SDK was, however, still required to build some of the DirectShow samples.[16]

Since November 2007, DirectShow APIs are part of the Windows SDK. It includes several new enhancements, codecs and filter updates such as the Enhanced Video Renderer (EVR) and DXVA 2.0 (DirectX Video Acceleration).[17]

Architecture

edit

DirectShow divides a complex multimedia task (e.g. video playback) into a sequence of fundamental processing steps known as filters.[18][19] Each filter – which represents one stage in the processing of the data – has input and/or output pins that may be used to connect the filter to other filters. The generic nature of this connection mechanism enables filters to be connected in various ways so as to implement different complex functions.[20] To implement a specific complex task, a developer must first build a filter graph by creating instances of the required filters, and then connecting the filters together.[21]

There are three main types of filters:

Source filters
These provide the source streams of data. For example, reading raw bytes from any media file.
Transform filters
These transform data that is provided from other filter's output. For example, doing a transform such as adding text on top of video or uncompressing an MPEG frame.
Renderer filters
These render the data. For example, sending audio to the sound card, drawing video on the screen or writing data to a file.

During the rendering process, the filter graph searches the Windows Registry for registered filters and builds its graph of filters based on the locations provided. After this, it connects the filters together, and, at the developer's request, executes (i.e., plays, pauses, etc.) the created graph.[citation needed] DirectShow filter graphs are widely used in video playback (in which the filters implement functions such as file parsing, video and audio demultiplexing, decompressing and rendering) as well as for video and audio recording, editing, encoding, transcoding or network transmission of media. Interactive tasks such as DVD navigation may also be controlled by DirectShow.[citation needed]

 
Filter graph of an mp3 file, as rendered by the DirectShow sample in GraphEdit, an application with a GUI for DirectShow used to visually build and test filter graphs.[22] In this picture the boxes represent filters and the grey dots appearing on the sides of the filters represent pins.

In the above example, from left to right, the graph contains a source filter to read an MP3 file, stream splitter and decoder filters to parse and decode the audio, and a rendering filter to play the raw audio samples. Each filter has one or more pins that can be used to connect that filter to other filters. Every pin functions either as an output or input source for data to flow from one filter to another. Depending on the filter, data is either "pulled" from an input pin or "pushed" to an output pin in order to transfer data between filters. Each pin can only connect to one other pin and they have to agree on what kind of data they are sending.[citation needed]

Most filters are built using a set of C++ classes provided in the DirectShow SDK, called the DirectShow Base Classes.[citation needed] These handle much of the creation, registration and connection logic for the filter.[23] For the filter graph to use filters automatically, they need to be registered in a separate DirectShow registry entry as well as being registered with COM. This registration can be managed by the DirectShow Base Classes. However, if the application adds the filters manually, they do not need to be registered at all.[citation needed] Unfortunately, it is difficult to modify a graph that is already running. It is usually easier to stop the graph and create a new graph from scratch. Starting with DirectShow 8.0, dynamic graph building, dynamic reconnection, and filter chains were introduced to help alter the graph while it was running.[24] However, many filter vendors ignore this feature, making graph modification problematic after a graph has begun processing.[citation needed]

Although DirectShow is capable of dynamically building a graph to render a given media type, in certain instances it is difficult for developers to rely on this functionality and they need to resort to manually building filter graphs if the resulting filter graph is variable.[citation needed] It is possible for filter graphs to change over time as new filters are installed on the computer.[citation needed]

Features

edit

By default, DirectShow includes a number of filters for decoding some common media file formats such as MPEG-1, MP3, Windows Media Audio, Windows Media Video, MIDI, media containers such as AVI, ASF, WAV, some splitters/demultiplexers, multiplexers, source and sink filters, some static image filters, some video acceleration, and minimal digital rights management (DRM) support.[25][26] DirectShow's standard format repertoire can be easily expanded by means of a variety of filters, enabling DirectShow to support virtually any container format and any audio or video codec. For example, filters have been developed for Ogg Vorbis, Musepack, and AC3, and some codecs such as MPEG-4 Advanced Simple Profile, AAC, H.264, Vorbis and containers MOV, MP4 are available from 3rd parties like ffdshow, K-Lite, and CCCP.[27][28][29] Incorporating support for additional codecs such as these can involve paying the licensing fees to the involved codec technology developer or patent holder.[citation needed] Finally, there are "bridge" filters that simultaneously support multiple formats, as well as functions like stream multiplexing, by exposing the functionality of underlying multimedia APIs such as VLC.[30]

The amount of work required to implement a filter graph depends on several factors. In the simplest case, DirectShow can create a filter graph automatically from a source such as a file or URL.[31] If this is not possible, the developer may be able to manually create a filter graph from a source file, possibly with the addition of a custom filter, and then let DirectShow complete the filter graph by connecting the filters together. At the next level, the developer must build the filter graph from scratch by manually adding and connecting each desired filter. Finally, in cases where an essential filter is unavailable, the developer must create a custom filter before a filter graph can be built.[citation needed]

Unlike the main C API of QuickTime where it is necessary to call MoviesTask in a loop to load a media file, DirectShow handles all of this in a transparent way. It creates several background threads that smoothly play the requested file or URL without much work required from the programmer.[citation needed] Also in contrast to QuickTime, nothing special is required for loading a URL instead of a local file on disk – DirectShow's filter graph abstracts these details from the programmer, although recent developments in QuickTime (including an ActiveX control) have reduced this disparity.[citation needed]

DirectShow Editing Services

edit

DirectShow Editing Services (DES), introduced in DirectX 8.0/Windows XP is an API targeted at video editing tasks and built on top of the core DirectShow architecture. DirectShow Editing Services was introduced for Microsoft's Windows Movie Maker.[1] It includes APIs for timeline and switching services, resizing, cropping, video and audio effects, as well as transitions, keying, automatic frame rate and sample rate conversion and such other features which are used in non-linear video editing allowing creation of composite media out of a number of source audio and video streams. DirectShow Editing Services allow higher-level run-time compositing, seeking support, and graph management, while still allowing applications to access lower-level DirectShow functions.

While the original API is in C++, DirectShow Editing Services is accessible in any Microsoft .NET compatible language including Microsoft Visual C# and Microsoft Visual Basic by using a third-party code library called "DirectShowNet Library".[32] Alternatively, the entire DirectShow API, including DirectShow Editing Services, can be accessed from Borland Delphi 5, 6 and 7, C++ Builder 6, and from later versions with a few minor modifications, using a third party software library called "DSPack".[33]

As of March, 2012[34] (and, apparently as early as 2009[35]), Microsoft has stated that the DirectShow Editing Services "API is not supported and may be altered or unavailable in the future."

Video rendering filters

edit

Originally, in Windows 9x, DirectShow used the Video Renderer filter. This drew the images using DirectDraw 3, but could also fall back to GDI or overlay drawing modes in some circumstances (depending upon the visibility of the video window and the video card's capabilities).[14][36] It had limited access to the video window.[citation needed] Video for Windows had been plagued with deadlocks caused by applications' incorrect handling of the video windows, so in early DirectShow releases, the handle to the playback window was hidden from applications. There was also no reliable way to draw caption text or graphics on top of the video.[citation needed]

DirectShow 6.0, released as part of DirectX Media introduced the Overlay Mixer renderer designed for DVD playback and broadcast video streams with closed captioning and subtitles.[citation needed] The Overlay Mixer uses DirectDraw 5 for rendering.[14] Downstream connection with the Video Renderer is required for window management.[citation needed] Overlay Mixer also supports Video Port Extensions (VPE), enabling it to work with analog TV tuners with overlay capability (sending video directly to a video card via an analog link rather than via the PCI bus). Overlay Mixer also supports DXVA connections.[citation needed] Because it always renders in overlay, full-screen video to TV-out is always activated.

Starting with Windows XP, a new filter called the Video Mixing Renderer 7 (VMR-7 or sometimes just referred to as VMR) was introduced. The number 7 was because VMR-7 only used DirectDraw version 7 to render the video and did not have the option to use GDI drawing. The main new feature of VMR-7 was the ability to mix multiple streams and graphics with alpha blending, allowing applications to draw text and graphics over the video and support custom effects.[37] It also featured a "windowless mode" (access to the composited image before it is rendered) which fixed the problems with access to the window handle.[38]

DirectX 9 introduced VMR-9, which is included in Windows XP SP2 and newer.[39] This version uses Direct3D 9 instead of DirectDraw, allowing developers to transform video images using the Direct3D pixel shaders.[40] It is available for all Windows platforms as part of the DirectX 9 redistributable.[citation needed] As VMR-7 it provides a Windowless Mode. However, unlike Overlay mixer or VMR-7 it does not support video ports.[41] Using the /3GB boot option may cause VMR-9 to fail.[42]

Windows Vista and Windows 7 ship with a new renderer, available as both a Media Foundation component and a DirectShow filter, called the Enhanced Video Renderer (EVR).[43] EVR is designed to work with Desktop Window Manager and supports DXVA 2.0, which is available on Windows Vista and Windows 7.[44][citation needed] It offers better performance and better quality according to Microsoft.[45]

Reception

edit

Awards

edit

On January 8, 2007. Microsoft received the Emmy award for Streaming Media Architectures and Components at the 58th Annual Technology & Engineering Emmy Awards.[46]

Simplicity

edit

Commanding DirectShow to play a file is a relatively simple task. However, while programming more advanced customizations, such as commanding DirectShow to display certain windows messages from the video window or creating custom filters, many developers complain of difficulties.[47][48] It is regarded as one of Microsoft's most complex development libraries/APIs.[49][citation needed]

Developers rarely create DirectShow filters from scratch. Rather, they employ DirectShow Base Classes.[citation needed] The Base Classes can often simplify development, allowing the programmer to bypass certain tasks. However, the process may remain relatively complex;[citation needed] the code found in the Base Classes is nearly half the size of the entire MFC library.[citation needed] As a result, even with the Base Classes, the number of COM objects that DirectShow contains often overwhelms developers.[citation needed][weasel words] In some cases, DirectShow's API deviates from traditional COM rules, particularly with regard to the parameters used for methods.[citation needed][weasel words] To overcome their difficulties with DirectShow's unique COM rules, developers often turn to a higher level API that uses DirectShow, notably, Windows Media Player SDK, an API provides the developer with an ActiveX Control that has fewer COM interfaces to deal with.[citation needed][weasel words]

Although DirectShow is capable of dynamically building a graph to render a given media type, in certain instances it is difficult for developers to rely on this functionality and they need to resort to manually building filter graphs if the resulting filter graph is variable.[citation needed] It is possible for filter graphs to change over time as new filters are installed on the computer.[citation needed]

Codec hell

edit

Codec hell (a term derived from DLL hell) is when multiple DirectShow filters conflict for performing the same task. A large number of companies now develop codecs in the form of DirectShow filters, resulting in the presence of several filters that can decode the same media type.[50][51] This issue is further exacerbated by DirectShow's merit system, where filter implementations end up competing with one another by registering themselves with increasingly elevated priority.[51]

Microsoft's Ted Youmans explained that "DirectShow was based on the merit system, with the idea being that, using a combination of the filter’s merit and how specific the media type/sub type is, one could reasonably pick the right codec every time. It wasn't really designed for a competing merit nuclear arms race."[51]

A tool to help in the troubleshooting of "codec hell" issues usually referenced is the GSpot Codec Information Appliance, which can be useful in determining what codec is used to render video files in AVI and other containers.[52] GraphEdit can also help understanding the sequence of filters that DirectShow is using to render the media file. Codec hell can be resolved by manually building filter graphs, using a media player that supports ignoring or overriding filter merits, or by using a filter manager that changes filter merits in the Windows Registry.[22]

End-user tools

edit

DirectShow, being a developer-centric framework and API, does not directly offer end-user control over encoding content, nor does it incorporate a user interface for encoding using installed codecs or to different formats; instead, it relies on developers to develop software using the API. In contrast, other multimedia frameworks such as QuickTime or Video for Windows allow end-users to perform basic video-related tasks such as re-encoding using a different codec and editing files and streams.[citation needed] The convenience offered by an end-user GUI is apparent since the AVI format and codecs used by Video for Windows still remain in use, for example VirtualDub.

See also

edit

References

edit
  1. ^ a b c Blome, Michael; Wasson, Mike (July 2002). "Core Media Technology in Windows XP Empowers You to Create Custom Audio/Video Processing Components". MSDN Magazine. Microsoft. Retrieved 2007-04-02.
  2. ^ "DirectShow". MSDN. 2012-12-01. Retrieved 2013-01-21.
  3. ^ a b "DirectShow documentation". MSDN Library. Microsoft. 2006-01-01. Retrieved 2006-01-01.
  4. ^ a b Weiss, Becky (30 May 2006). "Is the new Media Foundation a replacement of Direct Show?". icrosoft Developer Network forum.
  5. ^ "Migrating from DirectShow to Media Foundation". MSDN Library. Microsoft. July 2006. Retrieved 2010-08-17.
  6. ^ "DirectShow: Core Media Technology in Windows XP Empowers You to Create Custom Audio/Video Processing Components". docs.microsoft.com. 23 October 2019. Retrieved 31 March 2021.
  7. ^ B. Jones, Michael (July 1997). "The Microsoft Interactive TV System: An Experience Report". Microsoft Research. Microsoft. Archived from the original on 2007-05-16. Retrieved 2007-04-02.
  8. ^ "Microsoft Announces ActiveMovie" (Press release). Microsoft. 1996-03-05. Archived from the original on 2011-07-17. Retrieved 2006-12-31.
  9. ^ "Microsoft Internet Explorer 3.0 Beta Now Available" (Press release). Microsoft. 1996-05-29. Archived from the original on 2007-07-07. Retrieved 2006-12-31.
  10. ^ "Microsoft Unveils First Unified Multimedia API Strategy" (Press release). Microsoft. 1997-03-31. Archived from the original on 2007-10-24. Retrieved 2006-12-31.
  11. ^ "Microsoft and Progressive Networks Collaborate on Streaming Media" (Press release). Microsoft. 1997-07-21. Archived from the original on 2006-12-10. Retrieved 2006-12-31.
  12. ^ Eisler, Craig (2006-02-20). "DirectX Then and Now". Craig's Musings. Archived from the original on 2012-07-11. Retrieved 2010-08-17.
  13. ^ "Microsoft Releases DirectShow SDK - Universal Playback for DVD, Digital Video and Audio, and the Internet" (Press release). Microsoft. 1997-12-10. Archived from the original on 2010-02-12. Retrieved 2010-08-17.
  14. ^ a b c "DirectShow FAQ". MSDN Library. Microsoft. 2010-07-06. Retrieved 2010-08-17.
  15. ^ "FAQs about the DirectShow and Windows Media SDKs". MSDN Library. Microsoft. December 2001. Retrieved 2010-08-17.
  16. ^ "DirectX Samples". MSDN Library. Microsoft. 2005-09-11. Retrieved 2005-09-11.
  17. ^ "What's New in DirectShow Windows Vista". MSDN Library. Microsoft. 2006-11-08. Retrieved 2006-11-08.
  18. ^ Introduction to DirectShow application programming
  19. ^ DirectShow system overview
  20. ^ "About DirectShow filters". MSDN Library. Microsoft. 7 March 2008. Retrieved 2010-08-17.
  21. ^ "Overview of graph building". MSDN Library. Microsoft. 7 March 2008. Retrieved 2010-08-17.
  22. ^ a b "Overview of GraphEdit". MSDN Library. Microsoft. Retrieved 2010-08-17.
  23. ^ "DirectShow Base Classes". MSDN Library. Microsoft. 7 March 2008. Retrieved 2010-08-17.
  24. ^ "Dynamic Graph Building". MSDN Library. Microsoft. 7 March 2008. Retrieved 2010-08-17.
  25. ^ "Formats supported by default in DirectShow". MSDN Library. Microsoft. Retrieved 2005-09-11.
  26. ^ "Microsoft's next scapegoat - DirectShow!". retrakker. 2007-01-13. Retrieved 2010-08-18.
  27. ^ "illiminable Directshow Filters for Ogg Vorbis, Speex, Theora and FLAC". Archived from the original on 2006-02-24. Retrieved 2006-03-11.
  28. ^ "Monogram's DirectShow filters for Musepack, AAC and AMR". Archived from the original on 2011-09-26. Retrieved 2008-03-08.
  29. ^ "AC3Filter". Retrieved 2010-08-13.
  30. ^ Anderson, Dean; Lamberson, Jim (2007). "Using VideoLan VLC in DirectShow". An open source bridge from VLC to DirectShow. Retrieved 2008-02-15.
  31. ^ Thompson, Chris (2000-09-13). "DirectShow For Media Playback In Windows - Part III: Customizing Graphs". FLIPCODE.COM. Retrieved 2010-08-18.
  32. ^ "About DirectShowNet Library". DirectShowNet SourceForge Website. SourceForge. Retrieved 2010-08-18.
  33. ^ "The DSPack Project". Archived from the original on 2010-06-14. Retrieved 2010-08-18.
  34. ^ "DirectShow Editing Services". Retrieved 2012-04-09.
  35. ^ "Video editing like "DirectShow Editing Services"". Archived from the original on 2011-08-05. Retrieved 2012-04-09.
  36. ^ "DirectShow Video Renderer Filter". MSDN Library. Microsoft. Retrieved 2010-08-17.
  37. ^ "Video Mixing Renderer 7 (VMR-7)". Retrieved September 11, 2005.
  38. ^ "VMR Windowless Mode". MSDN Library. Microsoft. Retrieved 2005-09-11.
  39. ^ "Programming Guide for the DirectX SDK". msdn.microsoft.com. Archived from the original on 18 October 2010. Retrieved 11 January 2022.
  40. ^ "Video Mixing Renderer 9 (VMR-9)". MSDN Library. Microsoft. Retrieved 2005-09-11.
  41. ^ "Video Mixing Renderer Filter 9". MSDN Library. Microsoft. Retrieved 2005-09-11.
  42. ^ "Loss of DirectDraw Overlay and VMR9 after upgrade/Update".
  43. ^ "Enhanced Video Renderer". MSDN Library. Microsoft. Retrieved 2007-02-28.
  44. ^ Various. "DXVA 1.0 translator". MSDN Media Foundation forum. Microsoft. Archived from the original on 2009-01-13. Retrieved 2010-08-18.
  45. ^ "Choosing the Right Video Renderer". MSDN Library. Microsoft. Retrieved 2005-09-11.
  46. ^ "NATIONAL TELEVISION ACADEMY ANNOUNCES EMMY WINNING ACHIEVEMENTS: HONORS BESTOWED AT 58th ANNUAL TECHNOLOGY & ENGINEERING EMMY AWARDS" (PDF). Emmy Award. National Television Academy. January 2007. Archived from the original (PDF) on 2010-09-22. Retrieved 2007-01-08.
  47. ^ "Using DirectShow with Media Center Edition". /mceDirectShow. Archived from the original on 2010-11-21. Retrieved 2010-08-18.
  48. ^ Roushu, Kurifu (2001-03-13). "The Basics to using DirectShow". Gamedev.net. Archived from the original on 2012-02-08. Retrieved 2010-08-18.
  49. ^ Richards, Mark (2008-08-26). "Overview: What is the Augmented Reality Toolkit?". Archived from the original on 2008-07-19. Retrieved 2009-01-25.
  50. ^ "Codec Hell". HomeVideo101 Blog. 2007-06-22. Archived from the original on 2012-02-11. Retrieved 2010-08-18.
  51. ^ a b c Lanier, Chris (2005-01-27). "New Age DLL Hell? Hell Yes". Chris Lanier's Blog. MSMVPS.COM. Archived from the original on 2007-02-12. Retrieved 2007-02-27.
  52. ^ "The GSpot Codec Information Appliance". Retrieved 2010-08-18.
  • Pesce, Mark D. (2003). Programming Microsoft DirectShow for Digital Video and Television. Microsoft Press. ISBN 0-7356-1821-6.
edit
Retrieved from "https://en.wiki.x.io/w/index.php?title=DirectShow&oldid=1241386039"