Digital Image Inpainting

Digital image inpainting is the process of restoring small damaged areas of an image using information in nearby regions. Typical applications include retouching images by removing small details such as text or logos, or actual restoration such as the removal of digitized cracks, specks, or other defects.

In contrast to other image processing methods such as filtering, which typically treat an entire image in the same way, image inpainting makes a clear difference between damaged and undamaged areas. Damaged areas are marked as such in a mask, for example by human operators. The information in these areas is reconstructed using information from the remainder, i.e. the undamaged, areas.

Example

The image below (left) shows a photograph of Abraham Lincoln, originally taken on a glass plate which eventually cracked. We restored the image using inpainting. First, we manually created a mask outlining the crack (middle image). Using this mark and the original image, our inpainting method was used to digitally remove the crack (right).

Level-set based inpainting

We implemented our inpaining using a simple modification of the well-known Fast Marching Method (FMM) to compute level sets. Image pixels inside the mask are inpainted in increasing distance order from the mask's boundary, computed using the FMM. To inpaint a pixel, we consider the colors of the valid image pixels in a small radial neighborhood. These colors are combined so as to balance image smoothness but also propagate sharp image gradients (edges) at correct angles inside the mask. The implementation is described in this paper. The entire implementation is less than 600 lines of C++ code and runs in real-time on images of 1000 by 1000 pixels.

Examples

A vandalized fruit bowl image being inpainted

Street scene (top) inpainted with our method (middle) and the method of Bertalmio et al (bottom). The two inpaining methods yield very similar results. However, our method is around one order of magnitude faster and simpler to implement than the latter. Read the two papers and look at our code.

An inpainted photo of rock artist Tarja Turunen from Nightwish. The insets on the right show that, although the image is badly damaged, our inpainting does a good restoring job. We truly apologize for having vandalized the original image! Arguably we got bored with the ubiquitous Lena photo.

A more academic example showing how our inpainting is able to restore sharp edges under wide scratch damages.

Software

The implementation of our inpainting method is available here:

• Windows ready-to-run tool
• C++ source code (compilable with Visual Studio)

The software is simple to use: select an image file (BMP or PPM formats), a binary mask image (pixels to inpaint should be marked black), and visualize the result. The software is very easy to modify to build with other compilers, such as gcc.

Impact

Our method has been incorporated as one of the two inpainting methods in the well-known OpenCV open-source computer vision library!

Publication

See publication 36 available here.