The Timeline Model
A timeline is an object that Microsoft® DirectShow® Editing Services (DES) uses to represent a video editing project. This article explains timelines in conceptual terms, describing the various objects that a timeline contains.
This article contains the following sections.
Structure of a Timeline
A video production starts out as a collection of raw source files, such as video clips, sound files, and still images. The producer or video editor takes sections of those files—ten seconds from one clip, five seconds from another, thirty seconds from another. Then he or she arranges them in a particular sequence.
In the simplest case, the clips appear in a strictly linear order: Each clip is followed by the next. This basic arrangement forms a track. Thus, a simple production might contain just one video track and one audio track. More sophisticated productions use multiple tracks. For example, several audio tracks can be mixed together. The tracks might include audio effects, so that a track fades out while another fades in. With video, one track can transition to the next, using a fade or wipe. This is an example of compositing. Another example is a chroma key, in which the background of one track is keyed out and replaced by a different track. (An example of chroma keying is a weather forecaster placed in front of a satelite image.)
Thus, a video production contains source clips that are arranged in time (one clip followed by another) and in layers (one clip composited onto another). DES represents this organization using a tree:
- The leaf nodes are individual streams of audio or video, taken from media files. Each node is a source.
- A collection of sources with a uniform media type (audio or video) is a track.
- A collection of tracks is a composition. A composition is rendered as the composite of all the tracks it contains. Using an earlier example, the weather forecaster and the satellite map would reside on two separate tracks, within the same composition. Compositions can contain other compositions, which allows for complex arrangements of tracks.
- A top-level collection of compositions and tracks (all representing the same media type) is a group.
- A set of one or more groups forms a timeline. The timeline is the root node in the tree.
A timeline must contain at least one group. Each group represents a single stream in the final production. A typical production includes one video group and one audio group. Compositions are optional; they exist to provide more structure if needed.
The following illustration shows the timeline structure. This illustration shows child-parent relations in the tree, but not the time sequence.
How a Timeline Is Rendered
The tree diagram shown in the previous section does not indicate how the overlapping tracks and compositions are rendered. For this purpose, it is helpful to consider the timeline as a time sequence, rather than as a node structure. The following illustration shows a simple project with two video tracks and two audio tracks. The arrow at the top represents the direction of the timeline, starting from time 0.
First, consider the video group. Track 1 has a higher priority than Track 0. Wherever Track 1 contains a video source, that source obscures anything in Track 0. Where Track 1 is empty, Track 0 "shows through." When the video is rendered, it begins with Source A. Then, it cuts to Source C, obscuring the last part of Source A and the beginning of Source B. Finally, when Source C ends, it cuts to Source B (already in progress).
The audio group behaves differently: All tracks are simply mixed together. The audio begins with Source D, switches to a mix of Source D and Source E, and finishes with Source D by itself again.
Transitions
A transition is a way to segue from one video track to another, using a visual effect such as a fade or a wipe. In the preceding example, the video cuts are instantaneous. Now consider the same video group with a transition added to it:
The transition resides on Track 1. It represents a transition between that track and the layer above it (Track 0). At the start of the transition, the rendered output is entirely from Track 0 (source A). At the end, the output is entirely from Track 1 (source C). In between, the output transforms from Source A to Source C. In a fade transition, for example, one source progressively fades into another.
The rendered output is schematized along the bottom of the illustration. In the transition region, the video image transforms from Source A to Source C. There is still a jump cut from Source C to Source B, because the track does not contain a transition at that point.
Note that a transition has a particular direction. At the beginning of the transition, the video output is mostly from Source A. In the middle of the transition, the video is a mixture of the two sources. At the end, the output is mostly from Source B. By default, a transition goes from the track with lower priority to the one with higher priority, as shown. However, you can specify the opposite behavior. The following illustration shows an example of this situation. Track 0 fades to Track 1, and then Track 1 fades back to Track 0. Both transitions reside on Track 1.
Compositions and Layering
You can think of each video track as occupying its own layer. In a collection of tracks, the first track has the lowest priority (priority 0) and each subsequent track has a priority one level higher. At each layer, the source clips in that layer hide the source clips in the layers below it, unless that layer also contains a transition. Thus you can imagine DES making several passes when it renders:
First, it renders Track 0. There is nothing "under" Track 0, so empty regions are rendered as a solid black image. Transitions in this layer occur between the black image and Track 0 or vice versa. (For example, the video clip fades from black.) DES then lays Track 1 on top of Track 0, generating any transitions between the two tracks. The result is the composite of the two tracks. Next, it places Track 2 onto this composite. Transitions at this layer occur between the composite and Track 2. The process continues until the last (highest-priority) track is put down.
When several tracks are composited together, they behave like a single track (called a virtual track). In DES, the composition object encapsulates this behavior. You can insert multiple tracks into a composition, and then insert the composition into another composition (which might contain other tracks, or even other compositions). The first composition behaves as a single layer within the larger composition.
Complex results are possible with this model. For example, one video clip can wipe to a second clip, while at the same time the composite (the two clips plus the wipe) fades into a third clip.
Effects
Unlike transitions, effects apply to a single layer. Examples of video effects are blurring the video image, converting color video to monochrome, and rotating the image around an axis. Examples of audio effects are reverb and echo. You can apply an effect to an individual source, to a track, or to an entire composition.