Results: Output Quality

We successfully used Stan Birchfeld's KLT tracker with our implementation of affine transforms in MATLAB to stabilize the sample UAV video that Aswin provided us. The video is of six cars at the end of a runway with the plane slowly circling them. There is some jitter, and evidence of a couple of dropped frames. Our filter completely removes these, but it also eliminates the perspective change caused by the movement of the plane. This introduces considerable distortion after more than about 10 seconds. High pass filtering of the affine transformation series does remove the jitter while preserving the overall motion.

We wanted a more serious test of the jitter reduction, with more sudden motion. To do this we wrote some MATLAB code that takes an individual frame and generates a sequence of frames based on it, each with a random displacement from the original. The effect is that of a VERY jerky camera. The KLT-affine transform combination undoes this severe jitter quite nicely. We then superimposed a circular motion on top of the jitter to see if the filtered affine transformation series would preserve it while still removing the jitter. It does an acceptable job at this, although there are a few visible kinks.

Additional testing revealed that although the KLT tracker we used does a good job on tracking features through sudden translations, it cannot effectively deal with large sudden rotations. It loses track of all features in these cases. Hopefully this will not be an issue for our ultimate application, or we will be able to compensate for the rotation using additional input from gyros.

We also experimented with stabilizing jerky footage from movies, such as the opening scene to Saving Private Ryan. This works quite well! We invite you to test out our code on DVD-quality video and see what you think of the results. (At some point we plan to “stabilize” The Blair Witch Project so those of us prone to motion sickness can watch it without becoming ill.) Of course the output needs to be cropped somewhat to eliminate the black border caused by shifting the image: we cannot create data from nothing!

Results: Speed

Our code is not nearly fast enough for real-time use. Greyscale output at 640x480 resolution runs at about one-third of realtime, whereas color output at the same resolution is about one-tenth real time, on the 2GHz Intel Core2 Duo laptop used for testing. The biggest bottleneck right now seems to be in the interpolation used to assign pixel intensity values for the corrected frames. The KLT tracker itself is the next slowest component. Hopefully converting the code to C and/or offloading some of the work to the GPU will improve performance.

Content actions

Add module to:

My Favorites Login Required (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own Login Required (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Footer

This work is licensed by Robert Brockman, Jeffrey Bridge, and Stamatios Mastrogiannis under a Creative Commons Attribution License (CC-BY 3.0), and is an Open Educational Resource.

Last edited by Robert Brockman on Dec 21, 2009 3:11 pm -0600.