Generalized structure tensor

GST in 2D and locally orthogonal bases

Let the term image represent a function f ( ξ ( x , y ) , η ( x , y ) ) where x , y are real variables and ξ , η , and f , are real valued functions. GST represents the direction along which the image f can undergo an infinitesimal translation with minimal (total least squares) error, along the "lines" fulfilling the following conditions:

1. The "lines" are ordinary lines in the curvilinear coordinate basis ξ , η

which are curves in Cartesian coordinates as depicted by the equation above. The error is measured in the L 2 sense and the minimality of the error refers thereby to L2 norm.

2. The functions ξ ( x , y ) , η ( x , y ) constitute a harmonic pair, i.e. they fulfill Cauchy–Riemann equations,

Accordingly, such curvilinear coordinates ξ , η are locally orthogonal.

Then GST consists in

where 0 ≤ λ m i n ≤ λ m a x are the (infinitesimal) errors of translation in the best direction (designated by the angle θ ) and the worst direction (designated by θ + π / 2 ). The matrix I is the identity matrix.

Thereby, Cartesian Structure tensor is a special case of GST where ξ = x , and η = y .

Basic concept for its use in image processing and computer vision

Efficient detection of θ in images is possible by image processing for a pair ξ , η . Complex convolutions (or the corresponding matrix operations) and point-wise non-linear mappings are the basic computational elements of GST implementations. A total least square error estimation of 2 θ is then obtained along with the two errors, λ m a x and λ m i n . In analogy with the Cartesian Structure tensor, the estimated angle is in double angle representation, i.e. 2 θ is delivered by computations, and can be used as a shape feature whereas λ m a x − λ m i n alone or in combination with λ m a x + λ m i n can be used as a quality (confidence, certainty) measure for the angle estimation.

Logarithmic spirals, including circles, can for instance be detected by (complex) convolutions and non-linear mappings. The spirals can be in gray (valued) images or in a binary image, i.e. locations of edge elements of the concerned patterns, such as contours of circles or spirals, must not be known or marked otherwise.

Generalized structure tensor can be used as an alternative to Hough transform in image processing and computer vision to detect patterns whose local orientations can be modelled, for example junction points. The main differences comprise:

Physical and mathemical interpretation

The curvilinear coordinates of GST can explain physical processes applied to images. A well known pair of processes consist in rotation, and zooming. These are related to the coordinate transformation ξ = log ⁡ ( x 2 + y 2 ) and η = tan − 1 ⁡ ( x , y ) .

If an image f consists in iso-curves that can be explained by only $\xi$ i.e. its iso-curves consist in circles f ( ξ , η ) = g ( ξ ) , where g is any real valued differentiable function defined on 1D, the image is invariant to rotations (around the origin).

Zooming (comprising unzooming) operation is modeled similarly. If the image has iso-curves that look like a "star" or bicycle spokes, i.e. f ( ξ , η ) = g ( η ) for some differentable 1D function g then, the image f is invariant to scaling (w.r.t. the origin).

In combination,

f ( ξ , η ) = g ( cos ⁡ ( θ ) log ⁡ ( x 2 + y 2 ) + sin ⁡ ( θ ) tan − 1 ⁡ ( x , y ) )

is invariant to a certain amount of rotation combined with scaling, where the amount is precised by the parameter θ .

Analogously, the Cartesian structure tensor is a representation of a translation too. Here the physical process consists in an ordinary translation of a certain amount along x combined with translation along y ,

where the amount is specified by the parameter θ . Evidently θ here represents the direction of the line.

Generally, the estimated θ represents the direction (in ξ , η coordinates) along which infinitisemal translations leave the image invariant, in practice least variant. With every curvilinear coordinate basis pair, there is thus a pair of infinitesimal translators, a linear combination of which is a Differential operator. The latter are related to Lie algebra.

Miscelenous

Image in the context of the GST means both an ordinary image and an image neighborhood therein (local image), the context determining. For example, a photograph as well as any neighborhood of it are images.