Automatic Color Image Segmentation via Reversible Jump MCMC

Image segmentation is an important early vision task where pixels with similar features are grouped into homogeneous regions. As a matter of fact, the simplest statistical model for an image consists of the probabilities of pixel classes. The knowledge on the dependencies between nearby pixels can be modeled by a MRF. Such models are much more powerful, even if it is not easy to determine the values of the parameters which specify a MRF. If each pixel class is represented by a different model then the observed image may be viewed as a sample from a realization of an underlying label field. Unsupervised segmentation can therefore be treated as an incomplete data problem where the color values are observed, the label field is missing and the associated class model parameters, including the number of classes, need to be estimated.

Due to the difficulty of estimating the number of pixel classes (or clusters), unsupervised algorithms often assume that this parameter is known a priori. When the number of pixel classes is also being estimated, the unsupervised segmentation problem becomes a model selection problem over a combined model space.

Our approach consists of building a Bayesian color image model using a first order MRF. The observed image is represented by a mixture of multivariate Gaussian distributions while inter-pixel interaction favors similar labels at neighboring sites. In a Bayesian framework, we are interested in the posterior distribution of the unknowns given the observed image. Herein, the unknowns comprise the hidden label field configuration, the Gaussian mixture parameters, the MRF hyperparameter, and the number of mixture components (or classes). Then a MCMC algorithm is used to sample from the whole posterior distribution in order to obtain a MAP estimate via simulated annealing. However, classical MCMC methods are restricted to problems where the dimensionality of the parameter vector is fixed. Therefore, the estimation of the number of mixture components is not possible. Recently, a novel method, called Reversible Jump MCMC (RJMCMC), has been proposed by P. Green. This method makes it possible to construct reversible Markov chain samplers that jump between parameter subspaces of different dimensionality. It has also been applied to univariate Gaussian mixture identification, intensity based image segmentation, and computing medial axes of 2D shapes. Herein, RJMCMC allows us the direct sampling of the whole posterior distribution defined over the combined model space thus reducing the optimization process to a single simulated annealing run. Another advantage is that no coarse segmentation neither exhaustive search over a parameter subspace is required. Although we present the model in the case of 3 dimensional observations, it is straightforward to extend it to higher dimensions.

Results

We have conducted a test on the Berkeley segmentation dataset. The public benchmark data consists of grayscale and color human segmentations for 300 images. The images are divided into a training set of 200 images, and a test set of 100 images. Since our method doesn't need training, we have used the 100 color test images. The benchmark results of other segmentation algorithms are available at the Berkeley segmentation benchmark. The results of our algorithm with different beta parameter settings as well as results produced by JSEG, another unsupervised segmentation algorithm by Yining Deng and B.S.Manjunath, are available here.

Below we show some segmentation results obtained by the proposed method. Note that no user input was needed for these segmentations, only the input color image has been specified, the rest was done automatically by our algorithm.

Results on real color images