Read an Excerpt
Embedded Media Processing
By David J. Katz Rick Gentile Newnes
Copyright © 2006 Elsevier Inc.
All right reserved. ISBN: 978-0-08-045888-5
Chapter One
Embedded Media Processing
Why Are You Reading This Book?
Well, assuming that you find your pleasure reading elsewhere, we hope you're reading this book for at least one of the following reasons:
• You'd like to understand more about what embedded media processing entails, and what demands it places on designers who are more familiar with conventional systems based on microcontrollers (MCUs) or digital signal processors (DSPs).
• You need to design an embedded multimedia system and want to know how to choose the proper processor for the application.
• You've already chosen a processor, and you want to know how to start architecting your system so that you can avoid problems at the outset.
• You're in the middle of designing a multimedia system, but you've run into some snags with trying to get all of your data flows and memory accesses to line up properly.
• You're related to one of the authors.
In this book we concern ourselves with embedded media processing (EMP) systems—that is, applications involving large data blocks (whether image, video, audio, speech, or some combination of these), a need for signal processing, and (often) a real-time nature. By "embedded," we mean a specialized processor that's part of a larger system and is targeted for a specific application (as opposed to a personal computer, which is intended to do some of everything). A further requirement of many EMP systems is portability, so battery life is a key factor, and it's often necessary to reach a compromise between power dissipation and computational performance.
So What's All the excitement About embedded Multimedia Systems?
With the multimedia revolution in full swing, we're becoming accustomed to toting around cell phones, PDAs, cameras and MP3 players, concentrating our daily interactions into the palms of our hands. But given the usefulness of each gadget, it's surprising how often we upgrade to "the latest and greatest" device. This is, in part, due to the fact that the cell phone we bought last year can't support the new video clip playback feature touted in this year's TV ads.
After all, who isn't frustrated after discovering that his portable audio player gets tangled up over the latest music format? And which overworked couple has the time, much less the inclination, to figure out how to get the family vacation travelogue off their Mini-DV camcorder and onto a DVD or hard disk?
As Figure 1.1 implies, we've now reached the point where a single gadget can serve as a phone, a personal organizer, a camera, an audio player, and a web-enabled portal to the rest of the world.
But still, we're not happy.
Let's add a little perspective: we used to be satisfied just to snap a digital picture and see it on our computer screen. Just 10 years ago, there were few built-in digital camera features, the photo resolution was comparatively low, and only still pictures were an option. Not that we were complaining, since previously our only digital choice involved scanning 35-mm prints into the computer.
In contrast, today we expect multi-megapixel photos, snapped several times per second, that are automatically white-balanced and color-corrected. What's more, we demand seamless transfer between our camera and other media nodes, a feature made practical only because the camera can compress the images before moving them.
Clearly, consumer appetites demand steady improvement in the "media experience." That is, people want high-quality video and audio streams in small form factors, with low power requirements (for improved battery life) and at low cost. This desire leads to constant development of better compression algorithms that reduce storage requirements while increasing audio/video resolution and frame rates.
To a large extent, the Internet drives this evolution. After all, it made audio, images and streaming video pervasive, forcing transport algorithms to become increasingly clever at handling ever-richer media across the limited bandwidth available on a network. As a result, people today want their portable devices to be Net-connected, high-speed conduits for a never-ending information stream and media show. Unfortunately, networking infrastructure is upgraded at a much slower rate than bandwidth demands grow, and this underscores the importance of excellent compression ratios for media-rich streams.
It may not be readily apparent, but behind the scenes, processors have had to evolve dramatically to meet these new and demanding requirements. They now need to run at very high clock rates (to process video in real time), be very power efficient (to prolong battery life), and comprise very small, inexpensive single-chip solutions (to save board real estate and keep end products price-competitive). What's more, they need to be software-reprogrammable, in order to adapt to the rapidly changing multimedia standards environment.
A Simplified Look at a Media processing System
Consider the components of a typical media processing system, shown in Figure 1.2. Here, an input source presents a data stream to a processor's input interface, where it is manipulated appropriately and sent to a memory subsystem. The processor core(s) then interact with the memory subsystem in order to process the data, generating intermediate data buffers in the process. Ultimately, the final data buffer is sent to its destination via an output subsystem. Let's examine each of these components in turn.
Core Processing
Multimedia processing—that is, the actual work done by the media processor core-boils down into three main categories: format coding, decision operating and overlaying.
Software format coders separate into three classifications. Encoders convert raw video, image, audio and/or voice data into a compressed format. A digital still camera (DSC) provides a good example of an encoding framework, converting raw image sensor data into compressed JPEG format. Decoders, on the other hand, convert a compressed stream into an approximation (or exact duplicate) of the original uncompressed content. In playback mode, a DSC decodes the compressed pictures stored in its file system and displays them on the camera's LCD screen. Transcoders convert one media format into another one, for instance MP3 into Windows Media Audio 9 (WMA9).
Unlike the coders mentioned above, decision operators process multimedia content and arrive at some result, but do not require the original content to be stored for later retrieval. For instance, a pick-and-place machine vision system might snap pictures of electronic components and, depending on their orientation, size and location, rotate the parts for proper placement on a circuit board. However, the pictures themselves are not saved for later viewing or processing. Decision operators represent the fastest growing segment of image and video processing, encompassing applications as diverse as facial recognition, traffic light control, and security systems.
Finally, overlays blend multiple media streams together into a single output stream. For example, a time/date stamp might be instantiated with numerous views of surveillance footage to generate a composited output onto a video monitor. In another instance, graphical menus and icons might be blended over a background video stream for purposes of annotation or user input.
Considering all of these system types, the input data varies widely in its bandwidth requirements. Whereas raw audio might be measured in tens of kilobits/second (kb/s), compressed video could run several megabits per second (Mbps), and raw video could entail tens of megabytes per second (Mbytes/s). Thus, it is clear that the media processor needs to handle different input formats in different ways. That's where the processor's peripheral set comes into play.
Input/Output Subsystems—Peripheral Interfaces
Peripherals are classified in many ways, but a particularly useful generalization is to stratify them into functional groups like those in Table 1.1. Basically, these interfaces act to help control a subsystem, assist in moving and storing data, or enable connectivity with other systems or modules in an application.
Let's look now at some examples of each interface category.
Subsystem Control
Low-Speed Serial Interfaces
UART—Universal Asynchronous Receiver/Transmitter—As its name suggests, this full-duplex interface needs no separate clock or frame synchronization lines. Instead, these are decoded from the bit stream in the form of start bit, data bits, stop bits, and optional parity bits. UARTs are fairly low-speed (kbps to Mbps) and have high overhead, since every data word has control and error checking bits associated with it. UARTs can typically support RS-232 modem implementations, as well as IrDATM functionality for close-range infrared transfer.
SPI—Serial Peripheral Interface—This is a synchronous, moderate-speed (tens of Mbps), full-duplex master/slave interface developed by Motorola. The basic interface consists of a clock line, an enable line, a data input ("Master In, Slave Out") and a data output ("Master Out, Slave In"). SPI supports both multimaster and multislave environments. Many video and audio codecs have SPI control interfaces, as do many EEPROMs. We'll talk more about SPI in Chapter 5.
I2C—Inter-IC Bus—Developed by Philips, this synchronous interface requires only two wires (clock and data) for communication. The phase relationships between the two lines determines the start and completion of data transfer. There are primarily three speed levels: 100 kbps, 400 kbps and 3.4 Mbps. Like SPI, I2C is very commonly used for the control channel in video and audio converters, as well as in some ROM-based memories.
Programmable Timers—These multifunction blocks can generate programmable pulse-width modulated (PWM) outputs that are useful for one-shot or periodic timing waveform generation, digital-to-analog conversion (with an external resistor/capacitor network, for instance), and synchronizing timing events (by starting several PWM outputs simultaneously). As inputs, they'll typically have a width-capture capability that allows precise measurement of an external pulse, referenced to the processor's system clock or another timebase. Finally, they can act as event counters, counting external events or internal processor clock cycles (useful for operating system ticks, for instance).
Real-Time Clock (RTC)—This circuit is basically a timer that uses a 32.768 kHz crystal or oscillator as a time base, where every 215 ticks equals one second. In order to use more stable crystals, sometimes higher frequencies are employed instead; the most common are 1.048 MHz and 4.194 MHz. The RTC can track seconds, minutes, hours, days, and even years—with the functionality to generate a processor alarm interrupt at a particular day, hour, minute, second combination, or at regular intervals (say, every minute). For instance, a real-time clock might wake up a temperature sensor to sample the ambient environment and relay information back to the MCU via I/O pins. Then, a timer's pulse-width modulated (PWM) output could increase or decrease the speed of a fan motor accordingly.
Programmable Flags/GPIO (General Purpose Inputs/Outputs)—These all-purpose pins are the essence of flexibility. Configured as inputs, they convey status information from the outside world, and they can be set to interrupt upon receiving an edge-based or level-based signal of a given polarity. As outputs, they can drive high or low to control external circuitry. GPIO can be used in a "bit-banging" approach to simulate interfaces like I2 C, detect a keypress through a key matrix arrangement, or send out parallel chunks of data via block writes to the flag pins.
Watchdog Timer—This peripheral provides a way to detect if there's a system software malfunction. It's essentially a counter that is reset by software periodically with a count value such that, in normal system operation, it never actually expires. If, for some reason, the counter reaches 0, it will generate a processor reset, a nonmaskable interrupt, or some other system event.
(Continues...)
Excerpted from Embedded Media Processing by David J. Katz Rick Gentile Copyright © 2006 by Elsevier Inc.. Excerpted by permission of Newnes. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
