MIT develops algorithm that removes unwanted reflections from photos

If you’ve ever taken a photo from behind a window, you know you’re just as likely to get a good shot of yourself reflected in the glass as you are of what’s on the other side. Researchers from MIT have developed a process that could one day extract your reflection from such pictures, leaving only the intended subject on the other side of the glass.

This process actually relies upon the most severe sort of reflections as those are easier to pick out. Specifically, anything taken through double pane glass or very thick glass tends to have two overlapping reflections, one from the inner surface and one from the outer one. If you only have one reflection, it’s a very computationally complex problem for an algorithm to figure out what’s a reflection and what isn’t. With two identical offset images that are reflections, suddenly the computer has a point of reference.

The system created by the MIT team relies on finding the edges of the reflection by looking for a repeating offset pattern. The image is split into 8×8 blocks of pixels and calculates the correlation between the pixels. When the parts of the image comprising the reflection have been identified, the computer can then selectively tune the levels to make the reflection less pronounced. You can see an example of the results in the image above. It’s far from perfect, but this is just the first iteration of the technology.

This process currently requires the offset of the double reflection to be rather large to ensure the algorithm can recognize them as distinct shapes, but things might improve in the future. The team sees possible applications for this technology in consumer imaging technologies. Your phone might just know how to reduce reflections when you snap a photo through the window. These algorithms could also be of use in improving computer vision. Despite all the research that has gone into it, computers are still pretty bad at making sense of an image.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.