The Blahut–Arimoto algorithm, is often used to refer to a class of algorithms for computing numerically either the information theoretic capacity of a channel, or the rate-distortion function of a source. They are iterative algorithms that eventually converge to the optimal solution of the convex optimization problem that is associated with these information theoretic concepts.
For the case of channel capacity, the algorithm was independently invented by Arimoto and Blahut. In the case of lossy compression, the corresponding algorithm was invented by Richard Blahut. The algorithm is most applicable to the case of arbitrary finite alphabet sources. Much work has been done to extend it to more general problem instances.
Algorithm
Suppose we have a source
where