CS ILLINOIS

1. CS ILLINOIS DEPARTMENT OF COMPUTER SCIENCE Accelerating Motion Magnification Processing for Videos Mentor: Zicheng Liao • Mentees: ZubinPahuja, Allen Qiu Motivation Iterative Backstep Algorithm Motion magnification is a technique that amplifies barely perceptible movements in a video, making them visible to a human observer. This technique offers many potential applications, ranging from monitoring potential machine breakdowns in factories to finding more minute symptoms in healthcare patients. Our goal is to build upon ZichengLiao’s iterative backstep algorithm and implement a program that can amplify videos in real-time. • Cannot simply apply motion field i to frame i to determine the new frame i + 1 • Function is not onto – not every pixel in the new frame is guaranteed to have a value from the old frame • Instead we use an iterative backstepalgortihm • Start from each pixel in the new frame i + 1, then substitute in in the “source” pixel found in the old frame i • Finding the source pixel • Use x = x’ + αdx’ • x is the position of the new pixel in frame i + 1 • x’ and dx’ are the positions and motion vector from frame I • α is the degree of magnification we want to achieve • Finding x’ • Rearranging the formula yields x’ = x - αdx’ • Letting dx’ = dx: x’ = x - αdx • Now we know x’ and its corresponding dx’ • Substitute to find x’’ = x - αdx’ • Repeat • Through subsequent iterations with x’ of higher degree we obtain a better approximation of the “source” of x • We found 4 iterations to be optimal Progress • Program successfully applies the iterative backstep algorithm to generate amplified video clips • Motion fields are obtained before processing • Enabled extraction of the video frames and motion field data through OpenCV and Ce Liu’s previous work [1] • Made the processing of each frame independent, allowing the the possibility of parallelization in the future • Streamed the output frames in real-time after processing for each had finished • Not quite up to speed – only about 10 FPS Program Outline Step 1: Obtain motion fields between each frame Step 2: Calculate “source” of each pixel using iterative backstepping Step 3: Apply the source field to find each new pixel value dx x 0thbackstep Frame i Frame i + 1 Source Field i' x Frame i + 1 -αdx dx' x' 1stbackstep: x' = x - αdx dx'' x -αdx' x'' 2ndbackstep: x'' = x - αdx' Motion Field i … New Frame i + 1 Future Plans • Still have yet to meet our goal of magnification in real time • Current serial processing maxes out at 10 FPS • Processing is accomplished entirely on CPU • Our program has reimplemented the algorithm such that different frames can be processed independently • Parallelization using GPU power would be the next logical step • Several candidates in mind, but NVIDIA’s CUDA seems to be the most likely possibility • Since the finished frame will already be in the GPU we hope to display them immediately through an OpenGL buffer Acknowledgements References • Zicheng Liao • P.U.R.E. Committee [1] C. Liu. Beyond Pixels: Exploring New Representations and Applications for Motion Analysis. Doctoral Thesis. Massachusetts Institute of Technology. May 2009.