<< Chapter < Page Chapter >> Page >
Limited success was achieved, but more work is needed to get image stabilization to work in real time.

Results: output quality

We successfully used Stan Birchfeld's KLT tracker with our implementation of affine transforms in MATLAB to stabilize the sampleUAV video that Aswin provided us. The video is of six cars at the end of a runway with the plane slowly circling them.There is some jitter, and evidence of a couple of dropped frames. Our filter completely removes these, but it also eliminates theperspective change caused by the movement of the plane. This introduces considerable distortion after more than about 10 seconds. High pass filtering of the affine transformation series does remove the jitter while preserving the overall motion.

Uav footage stabilized with klt + affine transforms

Source footage provided by Aswin Sankaranarayanan.

We wanted a more serious test of the jitter reduction, with more sudden motion. To do this we wrote some MATLAB codethat takes an individual frame and generates a sequence of frames based on it, each with a random displacement from the original. Theeffect is that of a VERY jerky camera. The KLT-affine transform combination undoes this severe jitter quite nicely.We then superimposed a circular motion on top of the jitter to see if the filtered affine transformation series would preserve it whilestill removing the jitter. It does an acceptable job at this, although there are a few visible kinks.

Shifted image sequence stabilized with klt + filtered affine transforms

Additional testing revealed that although the KLT tracker we used does a good job on tracking features through sudden translations, it cannoteffectively deal with large sudden rotations. It loses track of all features in these cases. Hopefully this will not be an issue for our ultimate application, or we will be ableto compensate for the rotation using additional input from gyros.

We also experimented with stabilizing jerky footage from movies, such as the opening scene to Saving Private Ryan. This works quite well!We invite you to test out our code on DVD-quality video and see what you think of the results. (At some point we plan to “stabilize” TheBlair Witch Project so those of us prone to motion sickness can watch it without becoming ill.) Of course the output needs to be croppedsomewhat to eliminate the black border caused by shifting the image: we cannot create data from nothing!

Results: speed

Our code is not nearly fast enough for real-time use. Greyscale output at 640x480 resolution runs at about one-third of realtime,whereas color output at the same resolution is about one-tenth real time, on the 2GHz Intel Core2 Duo laptop used for testing. The biggest bottleneck right now seems to be in theinterpolation used to assign pixel intensity values for the corrected frames. The KLT tracker itself is the next slowest component.Hopefully converting the code to C and/or offloading some of the work to the GPU will improve performance.

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now




Source:  OpenStax, Fall 2009 elec301 group project report: video stabilization. OpenStax CNX. Dec 21, 2009 Download for free at http://cnx.org/content/col11152/1.1
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Fall 2009 elec301 group project report: video stabilization' conversation and receive update notifications?

Ask