Microsoft DirectX 8.0

Advanced Capture Topics

This article describes some advanced aspects of using Microsoft® DirectShow® in a capture application, although most capture applications do not require the information given here. Before reading this article, read How to Write a Capture Application.

Most capture applications can use the ICaptureGraphBuilder2 interface to build a capture graph. This interface correctly handles the majority of capture scenarios. However, if your application requires an unusually complicated graph, you might need to build the graph by hand, one filter at a time. If so, you must understand the details that ICaptureGraphBuilder2 would otherwise handle for you. Even if you construct your graph by hand, you might find some of the ICaptureGraphBuilder2 methods helpful, such as FindInterface and FindPin.

This article contains the following sections:

Repaint Events

If the user covers and uncovers the preview window, the video renderer receives a WM_PAINT message. By default, the video renderer requests a new frame so that it can repaint the window. If this occurs during file capture, it will corrupt your capture file and make it unusable. Therefore, if your capture graph writes data to a file, you must cancel the default handling for repaint events. Query the filter graph manager for the IMediaEvent interface, and call the IMediaEvent::CancelDefaultHandling method with the EC_REPAINT event code as the parameter.

For more information, see Event Notification in DirectShow.

Pin Categories

A capture filter can have multiple output pins, each with a different purpose. For example, some capture filters have separate pins for capture and preview. Others deliver closed-captioned data in addition to a primary video stream. The function of a pin is identified by a kernel-streaming property set. This article discusses the following pin categories:
Category GUIDDescription
PIN_CATEGORY_CAPTURECapture pin
PIN_CATEGORY_PREVIEWPreview pin
PIN_CATEGORY_VIDEOPORTVideo port (VP) pin
PIN_CATEGORY_CCClosed-captioned pin
PIN_CATEGORY_VBI Vertical blanking interval (VBI) pin
PIN_CATEGORY_VIDEOPORT_VBIVideo port pin for VBI

Every capture filter has at least one capture pin. It may also have a preview pin or a video port pin, but never both. Filters can have multiple capture pins and preview pins, each delivering a separate media type. Thus, a single filter might have video and audio capture pins, plus video and audio preview pins. Closed captioning and VBI are used in television and DVD.

To determine a pin's category, call the IKsPropertySet::Get method. The GUID of the property set is AMPROPSETID_Pin, and the property identifier is AMPROPERTY_PIN_CATEGORY. The following code example shows how to query the pin:

IPin *pPin; // Pointer to an output pin on a capture filter.
IKsPropertySet *pKs;
HRESULT hr;

hr = pPin->QueryInterface(IID_IKsPropertySet, (void **)&pKs);
if (SUCCEEDED(hr))
{
    GUID PinCategory;
    DWORD cbReturned;
    hr = pKs->Get(AMPROPSETID_Pin, AMPROPERTY_PIN_CATEGORY, NULL, 0, 
                        &PinCategory, sizeof(GUID), &cbReturned);
    if (SUCCEEDED(hr))
    {
        // PinCategory now contains the category GUID.
    }
    pKs->Release();
}

For more information about property sets, refer to the Microsoft® Windows® Driver Development Kit (DDK) documentation.

Using the Smart Tee Filter

If a filter provides separate capture and preview pins, you can capture from one while previewing from the other. However, even if the filter lacks a preview pin, you can do the same thing using the Smart Tee filter. This filter splits data from a capture pin into two identical streams, one for capture and one for preview.

The following illustration shows a graph with a Smart Tee filter.

Using the Smart Tee Filter

To compensate for previewing and capturing from one pin, the Smart Tee filter performs two optimizations:

The second feature requires some explanation: Live data always arrives slightly late at the renderer filter, because of latency in the graph. If preview data is time-stamped, the renderer might drop frames in order to catch up. However, no matter how many frames it drops, every frame is still late. Removing the time stamps prevents the renderer from dropping frames unnecessarily. For more information, see Time and Clocks in DirectShow.

Note  You can treat a video port pin as a kind of preview pin, so a filter with a VP pin does not need a Smart Tee filter. However, VP pins have a few special requirements. The next section describes these requirements.

Video Port Pins

A capture device with a hardware video port might use the video port extensions (VPE) in Microsoft® DirectX®. If so, the capture filter will have a video port (VP) pin. A VP pin must connect to an Overlay Mixer filter, and the Overlay Mixer must connect to the Video Renderer filter.

No video data travels from the VP pin through the filter graph. Instead, video frames are produced in hardware and sent directly to video memory. The VP pin allows control messages to be sent to the hardware. It is very important to connect the VP pin, even if you're only interested in capture functionality, since if you leave the pin unconnected, your capture graph will not work. This is different from preview pins, which can remain unconnected. Always connect the VP pin to input pin 0 on the Overlay Mixer.

The Video Renderer filter does not receive any video data from the Overlay Mixer. It controls the video window, however, so the filter graph must include it. You can make the video window a child of your application window by querying the filter graph manager for the IVideoWindow interface. For more information, see Setting the Video Window.

The following illustration shows a capture graph with a video port pin.

Filter Graph with Video Port Pin

The following code example connects a video port pin to the Overlay Mixer. It uses the ICaptureGraphBuilder2::FindPin method to find the capture filter's video port pin and the Overlay Mixer's input pin 0. (You could use the IBaseFilter::EnumPins method instead.) For brevity, the example omits error checking.

IPin *pPinOut,  // Video port pin on capture filter. 
     *pPinIn;   // Input pin on Overlay Mixer.
IBaseFilter *pOvMix = NULL;

// Find the video port pin.
pCGB->FindPin(
    pCaptureFilter,           // Pointer to capture filter.
    PINDIR_OUTPUT,            // Find an output pin.
    &PIN_CATEGORY_VIDEOPORT,  // Find a video port pin.
    NULL,                     // Any media type.
    TRUE,                     // Pin must be unconnected.
    0,                        // Retrieve first matching pin.
    &pPinOut                  // Address of pointer to pin.
);

// Create the overlay mixer.
CoCreateInstance(CLSID_OverlayMixer, NULL, CLSCTX_INPROC,
        IID_IBaseFilter, (void **)&pOvMix);

// Add it to the filter graph.
pGraph->AddFilter(pOvMix, L"Overlay Mixer");

// Retrieve input pin 0 on the overlay mixer.
pCGB->FindPin(pOvMix, PINDIR_INPUT, NULL, NULL, TRUE, 0, &pPinIn);

//Connect the two pins.
pGraph->Connect(pPinOut, pPinIn);

You can use similar code to connect the Overlay Mixer to the Video Renderer, or call IGraphBuilder::Render with the Overlay Mixer's output pin.

Closed Captioning

Closed-captioned pins come in several varieties:

The pin type determines the connection requirements:

Closed-Captioned Pin: For preview, connect this pin to the Line 21 Decoder filter and connect that filter to Pin 1 on the Overlay Mixer. (Pin 0 is reserved for the primary video stream.) If you call IGraphBuilder::Connect, the filter graph manager automatically adds the Line 21 Decoder. For file capture, connect to a mux filter, not the Overlay Mixer.

VBI Pin: Connect this pin to the Tee/Sink-to-Sink Converter filter (also called the VBI Infinite Tee). This filter splits the VBI data into separate streams. Connect this filter to the CC Decoder filter. For preview, connect the CC Decoder to the Overlay Mixer, as described previously. For file capture, connect it to a mux filter.

VBI Video Port Pin: Connect this pin to the VBI Surface Allocator filter (CLSID_VBISurfaces). This filter handles the Microsoft® DirectDraw® surfaces for the VBI data. If a filter has both a VBI pin and a VBI video port pin, connect both of them.

For preview, connect the capture filter's preview pin to Pin 0 on the Overlay Mixer. This applies to PIN_CATEGORY_PREVIEW pins as well as PIN_CATEGORY_VIDEOPORT pins. The Overlay Mixer overlays the closed captions onto the primary video.

The following diagram shows a preview graph with a VBI pin and a VBI video port pin.

Filter Graph with Closed Captioning

The Tee/Sink-to-Sink Converter and the CC Decoder filters are actually WDM mini-drivers, wrapped by the KSProxy filter. Therefore, you cannot create these filters by calling CoCreateInstance. Instead, use the system device enumerator to enumerate each filter's device category:

FilterCategory
Tee/Sink-to-Sink ConverterAM_KSCATEGORY_SPLITTER (WDM streaming tee/splitter)
CC DecoderAM_KSCATEGORY_VBICODEC (WDM streaming VBI codecs)

Within the category, test each filter's friendly name until you find a matching string. Then call IMoniker::BindToObject to create an instance of the filter.

The following example code creates a Tee/Sink-to-Sink Converter filter. For brevity, it omits error checking.

IBaseFilter *pFilter = NULL; // Pointer to receive the filter instance.

// Create the system device enumerator.
ICreateDevEnum *pDevEnum;
CoCreateInstance(CLSID_SystemDeviceEnum, NULL, CLSCTX_INPROC,
    IID_ICreateDevEnum, (void**)&pDevEnum);

// Create a class enumerator for the tee/splitter category.
IEnumMoniker *pEnum;
pDevEnum->CreateClassEnumerator(AM_KSCATEGORY_SPLITTER, &pEnum, 0);

// Enumerate devices within this category.
IMoniker *pMoniker;
ULONG cFetched;
while(pEnum->Next(1, &pMoniker, &cFetched) == S_OK)
{
    IPropertyBag *pBag;
    pMoniker->BindToStorage(0, 0, IID_IPropertyBag, (void **)&pBag);

    // Check the friendly name.
    VARIANT var;
    var.vt = VT_BSTR;
    pBag->Read(L"FriendlyName", &var, NULL);
    if (lstrcmpiW(var.bstrVal, L"Tee/Sink-to-Sink Converter") == 0)
    {
        // This is the right filter.
        pMoniker->BindToObject(0, 0, IID_IBaseFilter, (void**)&pFilter);
        SysFreeString(var.bstrVal);
        pBag->Release();
        pMoniker->Release();
        break;
    }
    SysFreeString(var.bstrVal);
    pBag->Release();
    pMoniker->Release();
}
pEnum->Release();
pDevEnum->Release();

VideoInfo2 Format Type

A preview pin might prefer a media type with a VIDEOINFOHEADER2 format. This format supports special features such as non-square pixels and interlaced video (field-based video, rather than frame-based video). The Overlay Mixer filter supports this format.

To take advantage of these features, do the following:

  1. Enumerate the pin's preferred media types, using the IPin::EnumMediaTypes method.
  2. Check the first media type in the enumeration sequence.
  3. If the formattype GUID is FORMAT_VideoInfo2, connect the pin to the Overlay Mixer. Then connect the Overlay Mixer to the video renderer. (See Video Port Pins.)

If you do not care about these features, you do not have to use the Overlay Mixer.

WDM Stream-Class Driver Filters

If a capture device uses a Windows Driver Model (WDM) driver, the graph might require certain filters upstream from the capture filter. These filters are called stream-class driver filters or WDM filters. They support additional functionality provided by the hardware. For example, a TV tuner card has functions for setting the channel. The corresponding filter is the TV Tuner filter, which exposes the IAMTVTuner interface. To make this functionality available to the application, you must connect the TV Tuner filter to the capture filter.

The ICaptureGraphBuilder2 interface provides the easiest way to add WDM filters to the graph. At some point while building the graph, call FindInterface or RenderStream. Either of these methods automatically locates the necessary WDM filters and connects them to the capture filter. The rest of this section describes how to add WDM filters manually. However, it is highly recommended that you call one of the ICaptureGraphBuilder2 methods instead.

The pins on a WDM filter support one or more mediums. A medium defines a method of communication, such as a bus. You must connect pins that support the same medium. Mediums are defined in DirectShow by the REGPINMEDIUM structure, which is equivalent to the KSPIN_MEDIUM structure used for kernel streaming drivers. To retrieve a pin's medium, call the IKsPin::KsQueryMediums method. This method returns a pointer to a block of memory that contains a KSMULTIPLE_ITEM structure, followed by zero or more REGPINMEDIUM structures. The clsMedium member of the REGPINMEDIUM structure specifies the class identifier (CLSID) for the medium.

The following helper function retrieves a pin's medium:

// Caller allocates pMedium.
HRESULT GetMedium(IPin *pPin, REGPINMEDIUM *pMedium)
{
    IKsPin *pKsPin = NULL;
    PKSMULTIPLE_ITEM pmi;

    HRESULT hr = pPin->QueryInterface(IID_IKsPin, (void **)&pKsPin);
    if (FAILED(hr)) 
        return hr;  // Pin does not support IKsPin.

    hr = pKsPin->KsQueryMediums(&pmi);
    pKsPin->Release();
    if (FAILED(hr))
        return hr;  // Pin does not support mediums.
    
    if (pmi->Count == 0) {
        CoTaskMemFree(pmi);
        return E_FAIL;  // Medium count is zero.
    }
    else {
        // Use pointer arithmetic to reference the medium structure.
        REGPINMEDIUM *pTemp = (REGPINMEDIUM*)(pmi + 1);
        memcpy(pMedium, pTemp, sizeof(REGPINMEDIUM));
        CoTaskMemFree(pmi);
        return S_OK;
    }        
}

Do not connect a pin if the medium has a CLSID of GUID_NULL or KSMEDIUMSETID_Standard. These are default values indicating that the pin does not support mediums.

Also, do not connect a pin unless the filter requires exactly one connected instance of that pin. Otherwise, your application might try to connect various pins that shouldn't have connections, which can cause the program to stop responding. To find out the number of required instances, retrieve the KSPROPERTY_PIN_NECESSARYINSTANCES property set, as shown in the following code example. (For brevity, this example does not test any return codes or release any interfaces. Your application should do both, of course.)

// Obtain the pin factory identifier.
IKsPinFactory *pPinFactory;
hr = pPin->QueryInterface(IID_IKsPinFactory, (void **)&pPinFactory);

ULONG ulFactoryId;
hr = pPinFactory->KsPinFactory(&ulFactoryId);

// Get the "instance" property from the filter.
IKsControl *pKsControl;
hr = pFilter->QueryInterface(IID_IKsControl, (void **)&pKsControl);

KSP_PIN ksPin;
ksPin.Property.Set = KSPROPSETID_Pin;
ksPin.Property.Id = KSPROPERTY_PIN_NECESSARYINSTANCES;
ksPin.Property.Flags = KSPROPERTY_TYPE_GET;
ksPin.PinId = ulFactoryId;
ksPin.Reserved = 0; 

KSPROPERTY ksProp;
ULONG ulInstances, bytes;
pKsControl->KsProperty((PKSPROPERTY)&ksPin, sizeof(ksPin), 
    &ulInstances, sizeof(ULONG), &bytes);

if (hr == S_OK && bytes == sizeof(ULONG)) 
{
    if (ulInstances == 1) 
    {
        // Filter requires one instance of this pin.
        // This pin is OK.
    } 
}

The following pseudocode is an extremely brief outline, showing how to find and connect the WDM filters. It omits many details, and is only meant to show the general steps your application would need to take.

Add supporting filters:
{
    foreach input pin:
        skip if (pin is connected)
    
        Get pin medium
        skip if (medium is GUID_NULL or KSMEDIUMSETID_Standard)
    
        Query filter for KSPROPERTY_PIN_NECESSARYINSTANCES property
        skip if (necessary instances != 1)

        Match an existing pin || Find a matching filter
}    

Match an existing pin:
{
    foreach filter in the graph
        foreach unconnected pin
            Get pin medium
            if (mediums match)
                connect the pins    
}

Find a matching filter:
{    
    Query the filter graph manager for IFilterMapper2.
    Find a filter with an output pin that matches the medium.
    Add the filter to the graph.
    Connect the pins.
    Add supporting filters. (Recursive call.)
}