Isotonic regression

Applications

Isotonic regression has applications in statistical inference, for example, to fit of an isotonic curve to mean experimental results when an order is expected. A benefit of isotonic regression is that it does not assume any form for the target function, such as linearity assumed by linear regression.

Another application is nonmetric multidimensional scaling, where a low-dimensional embedding for data points is sought such that order of distances between points in the embedding matches order of dissimilarity between points. Isotonic regression is used iteratively to fit ideal distances to preserve relative dissimilarity order.

Software for computing isotone (monotonic) regression has been developed for the R statistical package and the Python programming language.

Algorithms

In terms of numerical analysis, isotonic regression involves finding a weighted least-squares fit x ∈ R n to a vector a ∈ R n with weights vector w ∈ R n subject to a set of non-contradictory constraints of the kind x i ≤ x j . The usual choice for the constraints is x i ≤ x i + 1 , or in other words: every point must be at least as high as the previous point.

Such constraints define a partial ordering or total ordering and can be represented as a directed graph G = ( N , E ) , where N is the set of variables (observed values) involved, and E is the set of pairs (i, j) for each constraint x i ≤ x j . Thus, the IR problem corresponds to the following quadratic program (QP):

In the case when G = ( N , E ) is a total ordering, a simple iterative algorithm for solving this quadratic program is called the pool adjacent violators algorithm. Conversely, Best and Chakravarti (1990) studied the problem as an active set identification problem, and proposed a primal algorithm. These two algorithms can be seen as each other's dual, and both have a computational complexity of O(n).

Simply ordered case

To illustrate the above, let the x i ≤ x j constraints be x 1 ≤ x 2 ≤ … ≤ x n .

The isotonic estimator, g ∗ , minimizes the weighted least squares-like condition:

where A is the set of all piecewise linear, non-decreasing, continuous functions and f is a known function.