Machine epsilon gives an upper bound on the relative error due to rounding in floating point arithmetic. This value characterizes computer arithmetic in the field of numerical analysis, and by extension in the subject of computational science. The quantity is also called macheps or unit roundoff, and it has the symbols Greek epsilon
Contents
Values for standard hardware floating point arithmetics
The following values of machine epsilon apply to standard floating point formats:
Formal definition
Rounding is a procedure for choosing the representation of a real number in a floating point number system. For a number system and a rounding procedure, machine epsilon is the maximum relative error of the chosen rounding procedure.
Some background is needed to determine a value from this definition. A floating point number system is characterized by a radix which is also called the base,
Since machine epsilon is a bound for relative error, it suffices to consider numbers with exponent
Thus, the maximum spacing between a normalised floating point number,
Arithmetic model
Numerical analysis uses machine epsilon to study the effects of rounding error. The actual errors of machine arithmetic are far too complicated to be studied directly, so instead, the following simple model is used. The IEEE arithmetic standard says all floating point operations are done as if it were possible to perform the infinite-precision operation, and then, the result is rounded to a floating point number. Suppose (1)
By the meaning of machine epsilon, the relative error of the rounding is at most machine epsilon in magnitude, so:
where
Variant definitions
The IEEE standard does not define the terms machine epsilon and unit roundoff, so differing definitions of these terms are in use, which can cause some confusion.
The definition given here for machine epsilon is the one used by Prof. James Demmel in lecture scripts and his LAPACK linear algebra package, and by numerics research papers and some scientific computing software. Most numerical analysts use the words machine epsilon and unit roundoff interchangeably with this meaning.
The following different definition is much more widespread outside academia: Machine epsilon is defined as the smallest number that, when added to one, yields a result different from one. By this definition,
How to determine machine epsilon
Where standard libraries do not provide precomputed values (as <float.h> does with FLT_EPSILON
, DBL_EPSILON
and LDBL_EPSILON
for C and <limits> does with std::numeric_limits<T>::epsilon()
in C++), the best way to determine machine epsilon is to refer to the table, above, and use the appropriate pow formula. Computing machine epsilon is often given as a textbook exercise. The following examples compute machine epsilon in the sense of the spacing of the floating point numbers at 1 rather than in the sense of the unit roundoff.
Note that results depend on the particular floating-point format used, such as float, double, long double, or similar as supported by the programming language, the compiler, and the runtime library for the actual platform.
Some formats supported by the processor might not be supported by the chosen compiler and operating system. Other formats might be emulated by the runtime library, including arbitrary-precision arithmetic available in some languages and libraries.
In a strict sense the term machine epsilon means the 1+eps accuracy directly supported by the processor (or coprocessor), not some 1+eps accuracy supported by a specific compiler for a specific operating system, unless it's known to use the best format.
IEEE 754 floating-point formats have the property that, when reinterpreted as a two's complement integer of the same width, they monotonically increase over positive values and monotonically decrease over negative values (see the binary representation of 32 bit floats). They also have the property that 0 < |f(x)| < ∞, and |f(x+1) − f(x)| ≥ |f(x) − f(x−1)| (where f(x) is the aforementioned integer reinterpretation of x). In languages that allow type punning and always use IEEE 754-1985, we can exploit this to compute a machine epsilon in constant time. For example, in C:
This will give a result of the same sign as value. If a positive result is always desired, the return statement of machine_eps can be replaced with:
64-bit doubles give 2.220446e-16, which is 2−52 as expected.
Approximation
The following simple algorithm can be used to approximate the machine epsilon, to within a factor of two (one order of magnitude) of its true value, using a linear search.
epsilon = 1.0;while (1.0 + 0.5 * epsilon) ≠ 1.0: epsilon = 0.5 * epsilon