Skip to content Skip to navigation


You are here: Home » Content » High-Level Overview (Part 2)


Recently Viewed

This feature requires Javascript to be enabled.

High-Level Overview (Part 2)

Module by: Thomas Ladd, John Slack. E-mail the authors


Once all of the cups have been identified the replacement process begins by breaking up the area surrounding each cup into small boxes and examining each one individually. We then cycle through a bank of re-sized possible replacement images, breaking the corresponding area into the same sized chunks.  We calculate the cross correlation between each box, giving us an idea as to how well the replacement image matches the area surrounding the cup in the actual picture.  In order for the final product to look cohesive, the replacement image needs to “fit” the area surrounding the hole left by the cup.  The correlation between the cup itself and the object in the replacement image are inconsequential because the object will completely replace the cup after the replacement. Thus, only the correlation of the surrounding areas determine the selection of our replacement image.

Figure 1
Figure 1 (Picture 7.jpg)

The correlation of each pair of boxes is calculated in the same way as in the identification process, by taking the sum of products of each pixel. Once every pair of boxes has had its correlation calculated, the best match in the replacement bank is determined, and is placed in the position the detected red cup occupies. The best match does not necessarily have the most boxes with the highest correlation, or have the highest total correlation. Instead, some boxes are given a greater weight based on results from running an edge detector on each box. If the box contains more edges, it is given a greater weight. This is because it is most important to match edges when replacing the red solo cup, particularly hands. It is more obvious when edges do not match, and therefore edges are given a higher priority when selecting a replacement image. The replacement bank contains pictures of objects being held in hands in different orientations, so this methodology of weighting the boxes differently should lead to the most visually pleasing replacement possible.

Figure 2
Figure 2 (Picture 8.jpg)

The surrounding areas whose correlation was calculated are blurred and combined with the area surrounding the hole using a simple linear intensity blend to further facilitate blending of the replacement image with the image being filtered.

Figure 3
Figure 3 (Picture 9.jpg)

Content actions

Download module as:

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens


A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks