In what follows,
K
will denote the field of real or complex numbers. Let
K
m
×
n
denote the vector space containing all matrices with
m
rows and
n
columns with entries in
K
. Throughout,
A
∗
denotes the conjugate transpose of matrix
A
.
A matrix norm is a vector norm on
K
m
×
n
. That is, if
∥
A
∥
denotes the norm of the matrix
A
, then,
∥
A
∥
≥
0
∥
A
∥
=
0
if
A
=
0
∥
α
A
∥
=

α

∥
A
∥
for all
α
in
K
and all matrices
A
in
K
m
×
n
∥
A
+
B
∥
≤
∥
A
∥
+
∥
B
∥
for all matrices
A
and
B
in
K
m
×
n
.
Additionally, in the case of square matrices (thus, m = n), some (but not all) matrix norms satisfy the following condition, which is related to the fact that matrices are more than just vectors:
∥
A
B
∥
≤
∥
A
∥
∥
B
∥
for all matrices
A
and
B
in
K
n
×
n
.
A matrix norm that satisfies this additional property is called a submultiplicative norm (in some books, the terminology matrix norm is used only for those norms which are submultiplicative). The set of all
n
×
n
matrices, together with such a submultiplicative norm, is an example of a Banach algebra.
The definition of submultiplicativity is sometimes extended to nonsquare matrices, for instance in the case of the induced pnorm, where for
A
∈
K
m
×
n
and
B
∈
K
n
×
k
holds that
∥
A
B
∥
p
≤
∥
A
∥
p
∥
B
∥
q
. Here
∥
⋅
∥
p
and
∥
⋅
∥
q
are the norms induced from
K
m
, respectively
K
k
and p,q ≥ 1.
If vector norms on K^{m} and K^{n} are given (K is the field of real or complex numbers), then one defines the corresponding induced norm or operator norm on the space of mbyn matrices as the following suprema:
∥
A
∥
=
sup
{
∥
A
x
∥
:
x
∈
K
n
with
∥
x
∥
=
1
}
=
sup
{
∥
A
x
∥
∥
x
∥
:
x
∈
K
n
with
x
≠
0
}
.
The operator norm corresponding to the pnorm for vectors, p ≥ 1, is:
∥
A
∥
p
=
sup
x
≠
0
∥
A
x
∥
p
∥
x
∥
p
.
These are different from the entrywise pnorms, p ≥ 1, and the Schatten pnorms for matrices treated below, which are also usually denoted by
∥
A
∥
p
.
In the special cases of p = 1,2,∞, the norms can be computed or estimated by
∥
A
∥
1
=
max
1
≤
j
≤
n
∑
i
=
1
m

a
i
j

,
which is simply the maximum absolute column sum of the matrix.
∥
A
∥
∞
=
max
1
≤
i
≤
m
∑
j
=
1
n

a
i
j

,
which is simply the maximum absolute row sum of the matrix
∥
A
∥
2
≤
(
∑
i
=
1
m
∑
j
=
1
n

a
i
j

2
)
1
/
2
=
∥
A
∥
F
, where the right hand side is the Frobenius norm or
L_{2,2} norm. The equality holds if and only if the matrix A is a rankone matrix or a zero matrix.
For example, if the matrix A is defined by
A
=
[
−
3
5
7
2
6
4
0
2
8
]
,
then we have
∥
A
∥
1
=
max
(

−
3

+
2
+
0
,
5
+
6
+
2
,
7
+
4
+
8
)
=
max
(
5
,
13
,
19
)
=
19
and
∥
A
∥
∞
=
max
(

−
3

+
5
+
7
,
2
+
6
+
4
,
0
+
2
+
8
)
=
max
(
15
,
12
,
10
)
=
15.
In the special case of p = 2 (the Euclidean norm), the induced matrix norm is the spectral norm. The spectral norm of a matrix A is the largest singular value of A i.e. the square root of the largest eigenvalue of the positivesemidefinite matrix A^{∗}A:
∥
A
∥
2
=
λ
max
(
A
∗
A
)
=
σ
max
(
A
)
where A^{∗} denotes the conjugate transpose of A.
More generally, one can define the subordinate matrix norm on
K
m
×
n
induced by
∥
⋅
∥
α
on
K
n
, and
∥
⋅
∥
β
on
K
m
as:
∥
A
∥
α
,
β
=
max
x
≠
0
∥
A
x
∥
β
∥
x
∥
α
.
Subordinate norms are consistent with the norms that induce them, giving
∥
A
x
∥
β
≤
∥
A
∥
α
,
β
∥
x
∥
α
.
For
α
=
β
, any induced operator norm is a submultiplicative matrix norm since
∥
A
B
x
∥
≤
∥
A
∥
∥
B
x
∥
≤
∥
A
∥
∥
B
∥
∥
x
∥
and
max
∥
x
∥
=
1
∥
A
B
x
∥
=
∥
A
B
∥
.
Any induced norm satisfies the inequality
∥
A
r
∥
1
/
r
≥
ρ
(
A
)
,
where ρ(A) is the spectral radius of A. For a symmetric or hermitian matrix
A
, we have equality for the 2norm, since in this case the 2norm is the spectral radius of
A
. For an arbitrary matrix, we may not have equality for any norm. Take
A
=
[
0
1
0
0
]
,
the spectral radius of
A
is 0, but
A
is not the zero matrix, and so none of the induced norms are equal to the spectral radius of
A
.
Furthermore, for square matrices we have the spectral radius formula:
lim
r
→
∞
∥
A
r
∥
1
/
r
=
ρ
(
A
)
.
These vector norms treat an
m
×
n
matrix as a vector of size
m
n
, and use one of the familiar vector norms.
For example, using the pnorm for vectors, p ≥ 1, we get:
∥
A
∥
p
=
∥
v
e
c
(
A
)
∥
p
=
(
∑
i
=
1
m
∑
j
=
1
n

a
i
j

p
)
1
/
p
This is a different norm from the induced pnorm (see above) and the Schatten pnorm (see below), but the notation is the same.
The special case p = 2 is the Frobenius norm, and p = ∞ yields the maximum norm.
Let
(
a
1
,
…
,
a
n
)
be the columns of matrix
A
. The
L
2
,
1
norm is the sum of the Euclidean norms of the columns of the matrix:
∥
A
∥
2
,
1
=
∑
j
=
1
n
∥
a
j
∥
2
=
∑
j
=
1
n
(
∑
i
=
1
m

a
i
j

2
)
1
/
2
The
L
2
,
1
norm as an error function is more robust since the error for each data point (a column) is not squared. It is used in robust data analysis and sparse coding.
The
L
2
,
1
norm can be generalized to the
L
p
,
q
norm, p, q ≥ 1, defined by
∥
A
∥
p
,
q
=
(
∑
j
=
1
n
(
∑
i
=
1
m

a
i
j

p
)
q
/
p
)
1
/
q
When p = q = 2 for the
L
p
,
q
norm, it is called the Frobenius norm or the Hilbert–Schmidt norm, though the latter term is used more frequently in the context of operators on (possibly infinite dimensional) Hilbert space. This norm can be defined in various ways:
∥
A
∥
F
=
∑
i
=
1
m
∑
j
=
1
n

a
i
j

2
=
trace
(
A
∗
A
)
=
∑
i
=
1
min
{
m
,
n
}
σ
i
2
where A^{∗} denotes the conjugate transpose of A, σ_{i} are the singular values of A, and the trace function is used. The Frobenius norm is the Euclidean norm on
K
n
×
n
and comes from the Frobenius inner product on the space of all matrices.
The Frobenius norm is submultiplicative and is very useful for numerical linear algebra. This norm is often easier to compute than induced norms and has the useful property of being invariant under rotations, that is,
∥
A
∥
F
2
=
∥
A
R
∥
F
2
=
∥
R
A
∥
F
2
for any rotation matrix
R
. This property follows from the trace definition restricted to real matrices,
∥
A
R
∥
F
2
=
trace
(
R
T
A
T
A
R
)
=
trace
(
R
R
T
A
T
A
)
=
trace
(
A
T
A
)
=
∥
A
∥
F
2
and
∥
R
A
∥
F
2
=
trace
(
A
T
R
T
R
A
)
=
trace
(
A
T
A
)
=
∥
A
∥
F
2
where we have used the orthogonal nature of
R
, that is,
R
T
R
=
R
R
T
=
I
, and the cyclic nature of the trace,
trace
(
X
Y
Z
)
=
trace
(
Z
X
Y
)
. More generally the norm is invariant under a unitary transformation for complex matrices.
It also satisfies
∥
A
T
A
∥
F
=
∥
A
A
T
∥
F
=
∥
A
∥
F
2
and
∥
A
+
B
∥
F
2
=
∥
A
∥
F
2
+
∥
B
∥
F
2
+
2
⟨
A
,
B
⟩
F
where
⟨
A
,
B
⟩
F
is the Frobenius inner product.
The max norm is the elementwise norm with p = q = ∞:
∥
A
∥
max
=
max
i
j

a
i
j

.
This norm is not submultiplicative.
The Schatten pnorms arise when applying the pnorm to the vector of singular values of a matrix. If the singular values are denoted by σ_{i}, then the Schatten pnorm is defined by
∥
A
∥
p
=
(
∑
i
=
1
min
{
m
,
n
}
σ
i
p
)
1
/
p
.
These norms again share the notation with the induced and entrywise pnorms, but they are different.
All Schatten norms are submultiplicative. They are also unitarily invariant, which means that A = UAV for all matrices A and all unitary matrices U and V.
The most familiar cases are p = 1, 2, ∞. The case p = 2 yields the Frobenius norm, introduced before. The case p = ∞ yields the spectral norm, which is the matrix norm induced by the vector 2norm (see above). Finally, p = 1 yields the nuclear norm (also known as the trace norm, or the Ky Fan 'n'norm), defined as
∥
A
∥
∗
=
trace
(
A
∗
A
)
=
∑
i
=
1
min
{
m
,
n
}
σ
i
.
(Here
A
∗
A
denotes a positive semidefinite matrix
B
such that
B
B
=
A
∗
A
. More precisely, since
A
∗
A
is a positive semidefinite matrix, its square root is welldefined.)
A matrix norm
∥
⋅
∥
on
K
m
×
n
is called consistent with a vector norm
∥
⋅
∥
a
on
K
n
and a vector norm
∥
⋅
∥
b
on
K
m
if:
∥
A
x
∥
b
≤
∥
A
∥
∥
x
∥
a
for all
A
∈
K
m
×
n
,
x
∈
K
n
. All induced norms are consistent by definition.
A matrix norm
∥
⋅
∥
on
K
n
×
n
is called compatible with a vector norm
∥
⋅
∥
a
on
K
n
if:
∥
A
x
∥
a
≤
∥
A
∥
∥
x
∥
a
for all
A
∈
K
n
×
n
,
x
∈
K
n
. Induced norms are compatible by definition.
For any two matrix norms
∥
⋅
∥
α
and
∥
⋅
∥
β
, we have
r
∥
A
∥
α
≤
∥
A
∥
β
≤
s
∥
A
∥
α
for some positive numbers r and s, for all matrices A in
K
m
×
n
. In other words, all norms on
K
m
×
n
are equivalent; they induce the same topology on
K
m
×
n
. This is true because the vector space
K
m
×
n
has the finite dimension
m
×
n
.
Moreover, for every vector norm
∥
⋅
∥
on
R
n
×
n
, there exists a unique positive real number
k
such that
l
∥
⋅
∥
is a submultiplicative matrix norm for every
l
≥
k
.
A submultiplicative matrix norm
∥
⋅
∥
α
is said to be minimal if there exists no other submultiplicative matrix norm
∥
⋅
∥
β
satisfying
∥
⋅
∥
β
<
∥
⋅
∥
α
.
Let
∥
A
∥
p
once again refer to the norm induced by the vector pnorm (as above in the Induced Norm section).
For matrix
A
∈
R
m
×
n
of rank
r
, the following inequalities hold:
∥
A
∥
2
≤
∥
A
∥
F
≤
r
∥
A
∥
2
∥
A
∥
F
≤
∥
A
∥
∗
≤
r
∥
A
∥
F
∥
A
∥
max
≤
∥
A
∥
2
≤
m
n
∥
A
∥
max
1
n
∥
A
∥
∞
≤
∥
A
∥
2
≤
m
∥
A
∥
∞
1
m
∥
A
∥
1
≤
∥
A
∥
2
≤
n
∥
A
∥
1
.
Another useful inequality between matrix norms is
∥
A
∥
2
≤
∥
A
∥
1
∥
A
∥
∞
,
which is a special case of Hölder's inequality.