MMAPG - Digital Video Device
Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation
OS/2 Version 2.1 or later supports playback of software motion videos. The ability to create and record movies is provided by the Video IN product (part of the OS/2 BonusPak). The OS/2 digital video device (digitalvideo) provides file-format support through the multimedia input/output (MMIO) architecture. Several movie file formats including AVI, MPEG, FLC/FLI animation are supported. Several decompression types, like Ultimotion, Indeo 2.1, 3.1, and 3.2, FLC/FLI animation, and MPEG-1 are also supported. Files containing interleaved video and audio are supported, as well as video-only files.
Compression Formats
The digital video device provides video playback and recording support through its open compressor/decompressor (CODEC) architecture.
The following table describes the digital video compressors and decompressors (CODECs) available with OS/2. The CODECs are represented by unique FOURCC identifiers. A FOURCC is a 32-bit quantity representing a sequence of one to four ASCII alphanumeric characters (padded on the right with blank characters). For more information on CODEC procedures, see CODEC Procedures.
| CODEC | FOURCC | Description |
|---|---|---|
| Ultimotion | ULTI | Compressor/Decompressor (Real-time and Asymmetric compression) |
| Indeo 2.1 | rt21/RT21 | Compressor/Decompressor |
| Indeo 3.1 | v31/IV31 | Compressor/Decompressor (Real-time and Asymmetric compression) |
| Indeo 3.2 | v32/IV32 | Decompressor |
| Uncompressed (RAW) | DIB | Decompressor |
| FLI/FLC | FLIC | Decompressor |
| MPEG-1 | MPEG | Decompressor (RealMagic HW Decompression) |
About Ultimotion
Advances in microprocessor power, data storage, and compression technology have provided key technologies for creating and playing digital video data on personal computers. The high-capacity disk drives and CD-ROMs satisfy the large storage needs of digital video data. Additionally, today's more powerful microprocessors provide sufficient power to handle digital video data in real time. When these advances are combined with image compression techniques, the result is a powerful integration of video and the personal computer.
Several compression algorithms are currently in use throughout the industry. Some of these algorithms, like MPEG, use additional video hardware to compress and decompress the digitized video. Others are less numerically intensive and can be handled by software running on the main CPU and still maintain sufficient frame rates to provide motion. These are referred to as software-only algorithms or software motion video.
Ultimotion is IBM's technology for software motion video. It is a cross-platform algorithm that uses no hardware acceleration for capture or playback. The following sections describe Ultimotion, a single compression technique capable of providing a spectrum of quality levels from a single copy of the digital video data:
- Data Stream Capabilities
- Playback Characteristics
- Resolution Scalability
- Color Scalability
- Compression Ratios
- Symmetric Compression
- Asymmetric Compression
- Standard Ultimotion Movie
Data Stream Capabilities
Ultimotion is a playback-time scalable-video data stream. While the frame rate, output resolution, and color depth characteristics of a video are set when the video is created, the characteristics of a playback-scalable video are modified during playback utilizing the capabilities of the playback platform. These playback-platform capabilities depend on the type of microprocessor, display driver, video adapter, and data bandwidth available during playback.
Playback Characteristics
While Ultimotion is a playback-time scalable-video data stream, the magnitude that the data stream will 'scale' is determined by the amount of information that is encoded in the data stream when it was created; that is, the amount of data placed in the data stream at creation time determines the 'maximum' playback characteristics that a particular stream can achieve. In turn, the processing capabilities of the playback system determine how much of the data can be processed and presented during playback. Therefore, the playback characteristics of a given video are a function of the data put into the video by the author and the processing capabilities of the playback system.
The factors affecting the data stream at creation time are:
Resolution Width and height of video
Frame duration Frequency at which frames are to be created
I-frame rate Frequency at which reference frames are to occur
Data rate Average amount of data allowed for a second of video
Factors affecting the playback of a video are:
- Processing power of the CPU
- Throughput of data storage (for example, CD-ROM, hard disk, LAN)
- Efficiency of the display subsystem (such as the video adapter and display driver)
Resolution Scalability
Resolution of video determines how much spatial information is in a video file. Ultimotion compression algorithms organize this data so that it can be easily scaled up or down by factors of two as it is decompressed. Furthermore, as the data is decompressed, a sufficiently powered playback system can duplicate the data during output and display the video at four times its original size. This results in an effective output size larger than the input size. In this way, Ultimotion can be scaled down on systems incapable of processing the authored video resolution and scaled up on systems with more processing capability than the authored video requires.
Color Scalability
Ultimotion compression algorithms store images in 16-bit true color. This color is scaled down to the number of colors available on the playback system.
Compression Ratios
Ultimotion compressed frames use a coherent set of techniques for encoding a series of images. Different techniques are used for different purposes. Some represent detail very well while others represent large uniform areas with only a little data, and others fall in between but are very easily detected. Techniques can be mixed according to the needs of the image being compressed and are organized in an efficient manner for both compression and decompression. The result of mixing techniques is a data stream robust enough to be generated by software both asymmetrically and symmetrically.
Symmetric Compression
The Ultimotion compression techniques used on live video sources balance how well an image's detail is retained with how much time can be taken to analyze and compress the image. Symmetric Ultimotion compression uses the following items to determine the size and speed of symmetrically compressed Ultimotion videos:
- Image size
- Image quality (high, medium, low)
- Processor speed of the capture system
A 320 x 240 movie recorded at 15 frames per second at medium quality requires 9MB per minute of video. Reducing either the resolution, frame rate, or quality setting reduces the amount of storage required.
Asymmetric Compression
The Ultimotion compression techniques used during offline or asymmetric video recording take more time during the compression phase than it takes to decompress the video when the movie is played. By spending more time on compression, these techniques produce higher quality images and can compress the video data more effectively. Ultimotion asymmetric compression techniques produce movies stored using 320 x 240 resolution, 15 frame per second, and can be played from a CD. Movies made with these specifications require 9MB per minute. Reducing either the resolution, frame rate, or quality setting reduces the amount of storage required. Increasing these specifications require faster storage devices (for example, double speed CD-ROM or hard disk) and faster playback systems.
Standard Ultimotion Movie
Ultimotion compression algorithms use byte-oriented data structures and efficient data organization to provide software-only decompression. Since frame rate, resolution, and data rate are set when a movie is created, these settings determine the minimum platform required for playback. The 'standard Ultimotion movie' is defined as 320 x 240, 15 frames per second at a 150KB per second data rate. This movie can be played on at least a 25 MHz 80386 microprocessor and an SVGA display adapter. Computers with 33 MHz 80486 microprocessors are capable of displaying 320 x 240 resolution at 24 frames per second or 640 x 480 resolution at 15 frames per second. The frame rate increases as the the data rate of the source device and the machine speed increases. A frame rate of 30 frames per second is possible on a machine with a 66 MHz 80486 microprocessor.
Playback and Recording
OS/2 multimedia provides support to OS/2 Presentation Manager applications for controlling playback and recording of motion video. Playback of video can be performed in either a default window or an application window. Recording can be performed using video capture hardware devices with appropriate device driver support. The digital video device can also be used to capture still image (bitmap) data from video capture hardware or previously recorded motion video files.
This section describes digital video functions and provides examples using media control interface string commands.
Digital Video Windows
The digital video device provides two methods of displaying the video for a movie: a default window and an application-defined window.
Attention: To avoid unpredictable results, it is important that a Workplace Shell application does not create a PM window for digital video on the main thread. The application must start a separate thread, create the window on that thread, and then create a new message queue on that thread to service the window.
Default Window
The default window is used if no other window handle is specified with the window command. The parent of the default video window is HWND_DESKTOP unless otherwise specified with the open command. It provides a basic desktop window in which to display video that can be moved, sized, and minimized by the user. The default window is sized to the size of the video. The digital video device provides and manages this window for the application.
The default window has Normal Size, Double Size, and Half Size options on its settings menu. Selecting these options cause the window to be resized. The window can also be resized to other sizes, including icon size.
Application-Defined Window
An application-defined window can be used when the application requires more control over the window. The application can place video in its own client area or in a child window, add menus, and so on. If an application specifies a parent window handle when opening the device, it must close the logical video device before destroying the parent window.
If an application passes a window handle to the digital video device with MCI_WINDOW, it is essential that the application ensure that this window will receive all WM_REALIZEPALETTE messages sent to the message queue. If the window is a client-frame window (its parent has a style of CS_FRAME and the window has an ID of FID_CLIENT) or the child of a client window, it will receive this message automatically. If, however, the window has an ancestor that is not a client window and does not pass WM_REALIZEPALETTE to WinDefWindowProc, the window will not automatically receive the message. For example, WinDefDlgProc does not pass WM_REALIZEPALETTE on to child windows. So, if an ancestor of the window is a dialog window, the dialog procedure must explicitly pass the WM_REALIZEPALETTE message to the window. If the window does not receive this message, then incorrect colors will appear in video displayed in the window, whenever another application changes the system palette.
When an application-defined window is used to display video, the digital video device subclasses the window to ensure that video updating is maintained correctly. Subclassing the window does the following:
- Positions and sizes the window
- Prevents the window from being sized above the maximum supported size
- Modifies the tracking rectangle so the user cannot drag the size border beyond the screen boundaries
Opening the Device
The MCI_OPEN message is issued to create a device context of the digital video device. A default window is created and displayed when the device is opened if the device is cued for output or a movie is loaded. The video device directs its output to the default window. If a window is specified with the MCI_WINDOW message the video device directs its output to the window specified. For the convenience of applications that are using the string interface, some window-style control functions are also exported as flags to the MCI_WINDOW message.
Whether the default window or an application-defined window is used for output, the digital video device confines its output to the device coordinates of the specified window.
The default video window is invisible when the device is first opened. This allows the user or application to prepare the size, position, and contents of the window before it is displayed. It is created in a frame window, which can be sized, moved, maximized, and minimized. It is also created with an ideal aspect ratio in the center of the display and occupies approximately one quarter of the screen. As the user changes the size of the window, the digital video device scales the video image as required to maintain the video within the current coordinates. Maintaining a constant aspect ratio is the responsibility of the application program.
The parent of the default video window is HWND_DESKTOP unless otherwise specified in the MCI_OPEN message. The owner of the default video window is NULL. An application can specify a parent window handle for the default video window using hwndParent when the device is opened. This is the only window-related parameter that can be specified using media control interface commands that changes the behavior of the default video window.
Playing Motion Video Files
By default, playback of digital motion video is displayed in the window supplied by the digital video device. This window is created and displayed when the video is cued for output.
The following string commands illustrate playing an entire AVI format file. Because to and from flags are not specified, the file is played from the current position to the end of the file. When a motion video device element is opened, the current position in the media is the first playable area after any header or table of contents information.
open movie.avi type digitalvideo alias myvideo wait play myvideo notify close myvideo
Each frame in a motion video file has a number associated with it. From the perspective of the digital video device, each file is zero-based. That is, the first frame is frame 0, the second frame is frame 1, and so forth. This means the number of the last frame in a file is 1 less than the total number of frames in the file.
The current position always indicates the frame that is about to be displayed rather than the frame that is currently displayed.
When a play position is specified with the from flag, the actual position reached is accurate only to the nearest intracoded frame (I-frame). However, a position specified with the to flag is exact.
If you need to specify an exact position in the video file to play from, you can issue the seek command, which moves the current position in a file to an exact point. The following string commands illustrate moving the current position to frame 20 and then playing to frame 100.
open movie.avi type digitalvideo alias myvideo set myvideo time format frames wait seek myvideo to 20 wait play myvideo to 100 notify close myvideo
The cue command can also be used to seek and cue a particular frame. By specifying the show or noshow flag, you can control whether the video window will be displayed or hidden when the cue operation is performed. This can be useful for displaying video frames as a user moves the position slider, providing visual feedback of the location in the video a user is seeking to.
cue myvideo show to 20 wait
If a to position is not specified, the current frame is displayed and the media position will advance by one (frame).
Playing Hardware-Assisted MPEG Files
The RealMagic adapter enables hardware-assisted playback of MPEG files. MPEG video content is contained in an MPEG-specific file format. This file format, like AVI, interleaves audio and video data but is quite different from the tagged AVI format and requires a unique IOProc. The MPEG IOProc performs the file-format processing in the hardware-assisted environment, while decompression of both audio and video is performed by the hardware.
The digital video device provides normal-speed playback support (in the forward direction only) of MPEG files. The common file extension for MPEG files is .MPG. Seeking is supported, however seeking is an approximation to the nearest MPEG picture group. The size of a picture group is movie-dependent but is commonly 8 frames, so seeking takes place to within one-third of a second, in most cases.
Playing Animation Files
In addition to playing motion video with the digital video device, you can also play FLC and FLI animation files. FLC and FLI are the standard file formats for most animation tools. The FLI file format is most common. FLI is limited to a 320 x 200 display resolution and a custom 256-color palette. The FLC file format is an extension to FLI and allows a 640 x 480 resolution.
open cartoon.flc type digitalvideo alias animate play animate notify close animate
The FLC/FLI video files do not contain any audio data. However, the digital video device will search for an audio file (.WAV) with the same base name as the animation file in the same directory. If such a file exists, the audio is played with the animation file In the previous example, if a file named CARTOON.WAV exists, the audio file plays with the animation file and continues to play until CARTOON.FLC ends.
Seeking within a FLC/FLI file is not supported because these files contain only one initial I-frame. All other frames in the file are delta frames. All seek requests result in a seek to the beginning of the file.
Recording Motion Video
The digital video device supports real-time recording of motion video into AVI files. The supported compression algorithms for recording video are Ultimotion, Indeo 2.1, and Indeo 3.1. The default settings for real-time recording are 160 x 120 resolution, 15 frames per second, and Ultimotion compression type.
Recording into new and existing files is supported. The following example illustrates recording live video. As recording takes place, the digital audio and video data is stored in the temporary file created for the video device element. After the recording operation is complete, the device element is played back so it can be viewed before it is saved as a permanent file on disk.
To save the data as a video file, you can specify an existing file name or a new file name with the save command. If you indicate an existing file name, the data in the disk file is completely replaced by the data in the temporary file. You can also indicate that the file being saved is a video file; however, it is not necessary because this is the default.
In the example, monitoring is set on, so that the incoming video signal can be viewed in the default video window before it is recorded. Monitoring of live video can also be done without recording.
open digitalvideo alias myvideo wait set myvideo time format frames wait set myvideo monitor on wait record myvideo to 99 wait play myvideo wait save myvideo newvid.avi video wait close myvideo
Applications can specify that only video or only audio is to be recorded. In the following example, audio recording is turned off, so that only video will be recorded.
Specifying the cue input command causes the default window to become visible, so it is not necessary to set the monitoring function on. The cue command ensures the device is initialized and ready to record, preventing the possibility of losing any initial video when recording begins. As in the previous example, the device element is played back before it is saved as a file on disk.
open digitalvideo alias myvideo wait set myvideo record audio off wait set myvideo time format hms wait cue myvideo input wait record myvideo to 00:02:00 wait play myvideo wait save myvideo newvid.avi wait close myvideo
Image Support
The digital video device provides image support to applications that have the following requirements for digital video:
- Display an image of the current frame when the movie is stopped.
- Create and save an image in the specified format.
- Provide an image buffer to drag and drop cut-and-paste objects.
Capturing a Still Image
There are two functions applications can use to capture an image:
- MCI_GETIMAGEBUFFER
- MCI_CAPTURE
An application uses MCI_GETIMAGEBUFFER to capture an image when it wants to keep the image in an application buffer. The application can capture the image from either a movie file, or the buffer of a video capture card. This command message returns the image in the format specified by the application.
An application uses MCI_CAPTURE to capture an image from a movie file when it wants the image kept in a digital video device element.
Editing Operations
The digital video device supports the use of application buffers by editing commands to support drag and drop functions. The following flags are supported by the editing commands:
Message Flags
MCI_CUT MCI_TO_BUFFER
MCI_COPY MCI_FROM_BUFFER, MCI_TO_BUFFER
MCI_PASTE MCI_FROM_BUFFER, MCI_TO_BUFFER
To determine the buffer length for an editing operation, the application can issue MCI_CUT or MCI_COPY with either the MCI_FROM_BUFFER or MCI_TO_BUFFER flag set and a zero buffer length. The editing operation will not be performed, but the length of the buffer that is necessary for the operation will be returned in the buffer length field.
Setting the Tuner Device
With the appropriate hardware, the digital video device is able to monitor a TV channel. The WinTV card is an overlay card that has video capture support as well as television signal support. The monitor window displays the TV signal. The tuner is selected by setting the digital video device connector type to video in and the connector number to the tuner connector number of the adapter. The tuner connector number varies from adapter to adapter. On the WinTV adapter, this is connector number 2.
connector digitalvideo02 enable type video in number 2 wait
An application can use the settuner command to set the region, channel, and fine-tuning values. These flags are valid when used individually or in any combination. The digital video device uses these three values to calculate a frequency to send to the device. See the following example.
open digitalvideo03 alias mytuner wait settuner mytuner tv channel 3 region usa finetune plus 1 wait
The following example illustrates setting the frequency directly. Setting the frequency directly causes channel, region, and fine-tuning values to be ignored.
settuner mytuner frequency 80 wait
The region value must have a corresponding ASCII file (region.RGN) located in the \MMOS2\REGION subdirectory. When an application selects the region with the settuner command, the digital video device reads in the region file and fills a region array. The channel is used as an index into this array, which identifies the frequency. Fine-tuning is then added to the frequency. Region files are keyword-driven and have the following format:
[tuner] description=USA Cable lowchannel=2 highchannel=99 frequencies=0,0, 10050, -1, ... 29000
The description keyword identifies a descriptive string displayed on the corresponding Multimedia Setup page. The lowchannel and highchannel keywords indicate the range of channels available for the region. The frequencies correspond to the channel values, beginning with channel 0 and continuing to the highest available channel. The frequencies must be listed in columns of nine; otherwise, the region is invalid. A frequency value of 0 indicates the channel is out of range. A frequency value of -1 indicates a skipped channel, allowing the user to 'block' certain channels from being viewed.
Videodisc Command Messages
| Message | Description |
|---|---|
| MCI_OPEN | Initializes the videodisc device and creates a device context. |
| MCI_GETDEVCAPS | Retrieves the hardware capabilities of the specific videodisc player. |
| MCI_ESCAPE | Sends custom, device-dependent information or commands directly to the media driver. |
| MCI_CUE | Cues the device to a specific position to ensure minimum delay when a subsequent playback command is issued. |
| MCI_PLAY | Starts playing the videodisc. This message can be modified to:
|
| MCI_PAUSE | Suspends the current playback. On CAV discs, many players will freeze on the current frame. |
| MCI_RESUME | Resumes playback from a paused state while maintaining previously set parameters (like speed). |
| MCI_SEEK | Performs a high-speed search (forward or reverse) with video and audio muted. Can seek to specific positions, the start, or the end of the disc. |
| MCI_SET | Configures device and media attributes, including:
|
| MCI_STATUS | Queries the device for current information, such as:
|
| MCI_INFO | Retrieves the product name or model string of the peripheral device. |
| MCI_STOP | Terminates the current playback or seek operation. |
| MCI_SET_CUEPOINT | Defines a specific point in the media that triggers a notification when reached. |
| MCI_SET_POSITION_ADVISE | Requests periodic notifications of the current media position. |
| MCI_SPIN | Specifically controls the rotation of the disc (starting or stopping the motor). |
| MCI_STEP | Increments the playback by one or more units (usually frames) in either forward or reverse. |
| MCI_CLOSE | Releases the device context and shuts down the connection to the videodisc hardware. |
| MCI_CONNECTOR | Used to enable/disable specific hardware connectors, query their state, or identify their physical type. |