Salience-based Smoothing of 2D Shapes

Smoothing refers to the process of removing small-scale artifacts, such as noise, from images and shapes. This is a frequently-used step in most image and shape processing pipelines. Many smoothing methods exist, based on techniques such as anisotropic filtering, scale-space analysis, moment analysis, and Fourier and wavelet transforms.

However, most smoothing techniques cannot distinguish between noise and information below a certain spatial scale. Consider the noisy rectangle below (left): Typical filtering will remove the noisy dents and cusps, but also smooth out the corners (below, middle). What we usually want is a smooth rectangle with sharp corners (below, right). How to achieve this in a simple, robust, fast, and intuitive, way?

Skeletons to the rescue

Skeletons, or medial axes, are descriptors which capture the essential topological and geometrical properties of a shape, such as corners. Classical skeletons can be augmented with an importance metric which describes various geometric properties of the shape details they encode. They key idea is to refine this information to discriminate between small-scale noise details, such as cusps or dents, and important large-scale details, such as corners. For this, we define a new saliency metric which describes how important a detail is for an entire shape. Next, we use skeletons to compute this metric and also simplify (or smooth) the shape based on its values. The entire process can be done in literally a few tens of lines of code. For details, see our paper below.

Examples

The image below shows pairs of shapes with large-scale noise, which is hard to remove without excessive smoothing by classical filtering methods, and the results of our salience-based smoothing. Note how the important shape corners and dents (concavities) are preserved while smaller-scale details are entirely filtered away.

Image reconstruction

Is this all? Well, not quite. We can use precisely the same method also to reconstruct crisp, corner-preserving, images from highly noisy data. The figure below (left) shows a highly noisy image obtained from a typical brain CT segmentation. There are lots of holes and disconnected segments, and the exact outline of the shape is not clear. We can use our salience-based smoothing to both reconstruct and filter the image. This yields a clean, singly-connected, hole-free image with both sharp corners and smooth boundaries (below, right). Note that no additional steps such as pre-filtering, connected component detection, or similar, have been performed here: the salience-encoding skeletons perform both filtering and reconstruction in one single step.

More information

The entire salience-based smoothing method is described in paper 134 here. A ready-to-use implementation (NVIDIA CUDA based) of this method is available here.