We present a system including a novel component called programmable aperture and two associated post-processing algorithms for high-quality light field acquisition. The shape of the programmable aperture can be adjusted and used to capture light field at full sensor resolution through multiple exposures without any additional optics and without moving the camera. High acquisition efficiency is achieved by employing an optimal multiplexing scheme, and quality data is obtained by using the two postprocessing algorithms designed for self calibration of photometric distortion and for multi-view depth estimation. View-dependent depth maps thus generated help boost the angular resolution of light field. Various post-exposure photographic effects are given to demonstrate the effectiveness of the system and the quality of the captured light field.

Reducing LCD backlight can save power consumption of a portable device, but it also decreases the contrast and brightness of the images. Previous approaches adjust the backlight level frame by frame to reach a specified image quality level but do not optimize it. On the contrary, the proposed method adjusts the backlight to meet the target power level while maintaining the image quality. This is achieved by incorporating brightness compensation and local contrast enhancement depending on the given backlight level. Experimental results show that the proposed algorithm outperforms previous methods.

The electronic rolling shutter approach found in most low-end CMOS image sensors collects image data row by row, analogous to an open slit that scans over the image sequentially. Each row integrates light when the slit passes over it. Therefore, the scanlines of the image are not exposed at the same time. This sensor architecture creates a geometric distortion, known as the rolling shutter effect, for moving objects. We address this problem by using digital image processing techniques.

Digital image processing is an important technique to the consumer video electronics or other video capture devices. The digital image stabilization system is especially an important component used in the application such as security surveillance, military reconnaissance, and digital camera. The task of image sequence stabilization is accomplished by estimating and then compensating the global motion to remove the involuntary image movement caused by, for example unstable hand-shake or vibration. Our research targets to develop the fast and cost-effective image stabilization algorithm on the embedded system such as digital camera.

Our goal is to research on multimedia signal processing for home gateway in order to provide digital multimedia access in the home with excellent quality and complete digital right management. The milestone of the first year is to research on content-aware QoS and digital rights management system. The milestone of the second year is to research on complexity-aware streaming and digital rights management on MHP and mobile devices. The milestone of the final year is to research on the error-resilient video streaming for H.264, R-D optimization for complexity-aware encoder, system integration and performance evaluation.

Music plays an important role in human’s history, even more so in the digital age. Never before has such a large collection of music been created and accessed daily by people. As the amount of content continues to explode, the way music information is organized has to evolve in order to meet the ever increasing demand for easy and effective information access. Music classification and retrieval by emotion is a plausible approach, for it is content-centric and functionally powerful.

Video coding has become a key technology for a wide range of applications, from personal computers to television. It makes it possible and efficient for video storage and transmission. The latest developed video coding standard H.264 has emerged as a popular video coding technology for multimedia communications and consumer electronics as well. Our reseach topics on H.264 encoder optimization includes fast motion estimation and mode decision, rate control, and rate-distortion optimization.