In computer vision, maximally stable extremal regions (MSER) are used as a method of blob detection in images. This technique was proposed by Matas et al. to find correspondences between image elements from two images with different viewpoints. This method of extracting a comprehensive number of corresponding image elements contributes to the wide-baseline matching, and it has led to better stereo matching and object recognition algorithms.
Contents
Terms and Definitions
Image
-
S is totally ordered (total, antisymmetric and transitive binary relations≤ exist). - An adjacency relation
A ⊂ D × D is defined.
Region
(Outer) Region Boundary
Extremal Region
Maximally Stable Extremal Region Let
The equation checks for regions that remain stable over a certain number of thresholds. If a region
The concept more simply can be explained by thresholding. All the pixels below a given threshold are 'black' and all those above or equal are 'white'. Given a source image, if we generate a sequence of thresholded result images
Extremal regions
Extremal regions in this context have two important properties, that the set is closed under...
- continuous transformation of image coordinates. This means it is affine invariant and it doesn't matter if the image is warped or skewed.
- monotonic transformation of image intensities. The approach is of course sensitive to natural lighting effects as change of day light or moving shadows.
Advantages of MSER
Because the regions are defined exclusively by the intensity function in the region and the outer border, this leads to many key characteristics of the regions which make them useful. Over a large range of thresholds, the local binarization is stable in certain regions, and have the properties listed below.
Note, however, that detection of MSERs in a scale pyramid improves repeatability, and number of correspondences across scale changes.
Comparison to other region detectors
In Mikolajczyk et al., six region detectors are studied (Harris-affine, Hessian-affine, MSER, edge-based regions, intensity extrema, and salient regions). A summary of MSER performance in comparison to the other five follows.
Note however that this evaluation did not make use of multi-resolution detection, which has been shown to improve repeatability under blur.
MSER consistently resulted in the highest score through many tests, proving it to be a reliable region detector.
Implementation
The original algorithm of Matas et al. is
The component tree is the set of all connected components of the thresholds of the image, ordered by inclusion. Efficient (quasi-linear whatever the range of the weights) algorithms for computing it do exist. Thus this structure offers an easy way for implementing MSER.
More recently, Nister and Stewenius have proposed a truly (if the weight are small integers) worst-case
Robust wide-baseline algorithm
The purpose of this algorithm is to match MSERs to establish correspondence points between images. First MSER regions are computed on the intensity image (MSER+) and on the inverted image (MSER-). Measurement regions are selected at multiple scales: the size of the actual region, 1.5x, 2x, and 3x scaled convex hull of the region. Matching is accomplished in a robust manner, so it is better to increase the distinctiveness of large regions without being severely affected by clutter or non-planarity of the region's pre-image. A measurement taken from an almost planar patch of the scene with stable invariant description are called a 'good measurement'. Unstable ones or those on non-planar surfaces or discontinuities are called 'corrupted measurements'. The robust similarity is computed: For each
This algorithm can be tested here (Epipolar or homography geometry constrained matches): WBS Image Matcher
Use in Text Detection
The MSER algorithm has been used in text detection by Chen by combining MSER with Canny edges. Canny edges are used to help cope with the weakness of MSER to blur. MSER is first applied to the image in question to determine the character regions. To enhance the MSER regions any pixels outside the boundaries formed by Canny edges are removed. The separation of the later provided by the edges greatly increase the usability of MSER in the extraction of blurred text. An alternative use of MSER in text detection is the work by Shi using a graph model. This method again applies MSER to the image to generate preliminary regions. These are then used to construct a graph model based on the position distance and color distance between each MSER, which is treated as a node. Next the nodes are separated into foreground and background using cost functions. One cost function is to relate the distance from the node to the foreground and background. The other penalizes nodes for being significantly different from its neighbor. When these are minimized the graph is then cut to separate the text nodes from the non-text nodes. To enable text detection in a general scene, Neumann uses the MSER algorithm in a variety of projections. In addition to the greyscale intensity projection, he uses the red, blue, and green color channels to detect text regions that are color distinct but not necessarily distinct in greyscale intensity. This method allows for detection of more text than solely using the MSER+ and MSER- functions discussed above.