Thomas M. Boccuto

Abstract

Ordinary videos capture a surprising amount of hidden, visually imperceptible information. For instance, videos of peoples' faces may capture color changes in the skin and artery motion from heartbeats, while videos of mechanical systems can capture subtle vibrations indicating imminent failure. Algorithms can extract and exaggerate these signals for visualization on top of the original videos. In particular, Eulerian magnification algorithms sidestep the need to track hidden motions directly and instead devise multiscale bandpass filters to amplify signals in local spatial regions. In this work, we extend these techniques beyond color videos to geometric video data captured by 3D depth cameras such as the Microsoft Kinect. In our framework, we can spatially amplify a "bulging of the neck" during a heartbeat or the expansion of a chest/abdomen during a breath. We then exaggerate and display these signals as evolving 3D shapes. We explore pipelines based both on implicit and parameterized surface representations, and we discuss the merits, drawbacks, and challenges of both representations compared to ordinary color videos

Key contributions include:

• Development of spatial amplification techniques to highlight subtle geometric changes (e.g., neck bulging during heartbeats, chest expansion during breathing)
• Implementation of both implicit and parameterized surface representation pipelines
• Comparative analysis of these representations against traditional color video approaches
• Visualization of amplified signals as evolving 3D shapes