Dual norm

Definition

Let X and Y be topological vector spaces, and L ( X , Y ) be the collection of all bounded linear mappings (or operators) of X into Y . In the case where X and Y are normed vector spaces, L ( X , Y ) can be normed in a natural way.

When Y is a scalar field (i.e. Y = C or Y = R ) so that L ( X , Y ) is the dual space X ∗ of X , the norm on L ( X , Y ) defines a topology on X ∗ which turns out to be stronger than its weak-*topology.

Theorem 1: Let X and Y be normed spaces, and associate to each f ∈ L ( X , Y ) the number:

∥ f ∥ = sup { | f ( x ) | : x ∈ X , ∥ x ∥ ≤ 1 }

We first establish that L ( X , Y ) is bounded (using the triangle inequality), and complete (using Cauchy sequences) using our definition of ∥ f ∥ , thereby making L ( X , Y ) a normed space. If Y is a Banach space, so is L ( X , Y ) .

Proof:

1. A subset of a normed space is bounded if and only if it lies in some multiple of the unit sphere; thus ∥ f ∥ < ∞ for every f ∈ L ( X , Y ) if α is a scalar, then ( α f ) ( x ) = α ⋅ f x so that ∥ α f ∥ = | α | ∥ f ∥ The triangle inequality in Y shows that ∥ ( f 1 + f 2 ) x ∥ = ∥ f 1 x + f 2 x ∥ ≤ ∥ f 1 x ∥ + ∥ f 2 x ∥ ≤ ( ∥ f 1 ∥ + ∥ f 2 ∥ ) ∥ x ∥ ≤ ∥ f 1 ∥ + ∥ f 2 ∥ for every x ∈ X with ∥ x ∥ ≤ 1 . Thus ∥ f 1 + f 2 ∥ ≤ ∥ f 1 ∥ + ∥ f 2 ∥ If f ≠ 0 , then f x ≠ 0 for some x ∈ X ; hence ∥ f ∥ > 0 . Thus, L ( X , Y ) is a normed space.
2. Assume now that Y is complete, and that { f n } is a Cauchy sequence in L ( X , Y ) . Since and it is assumed that ∥ f n − f m ∥ → 0 as n and m tend to ∞ , { f n x } is a Cauchy sequence in Y for every x ∈ X . Hence exists. It is clear that f : X → Y is linear. If ε > 0 , ∥ f n − f m ∥ ∥ x ∥ ≤ ε ∥ x ∥ for sufficiently large n and m. It follows ∥ f x − f m x ∥ ≤ ε ∥ x ∥ for sufficiently large m. Hence ∥ f x ∥ ≤ ( ∥ f m ∥ + ε ) ∥ x ∥ , so that f ∈ L ( X , Y ) and ∥ f − f m ∥ ≤ ε . Thus f m → f in the norm of L ( X , Y ) . This establishes the completeness of L ( X , Y )

Theorem 2: Now suppose B is the closed unit ball of normed space X . Define

∥ x ∗ ∥ = sup { | ⟨ x , x ∗ ⟩ | : x ∈ B }

for every x ∗ ∈ X ∗

The second dual of a Banach space is an isometric isomorphism

The normed dual X ∗ of a Banach space X is also a Banach space, which means it has a normed dual, X ∗ ∗ , of its own.

By part (b) of Theorem 2, every x ∈ X defines a unique ϕ ∈ X ∗ ∗ by equation

⟨ x , x ∗ ⟩ = ⟨ x ∗ , ϕ x ⟩ ( x ∗ ∈ X ∗ ) ;

and

∥ ϕ x ∥ = ∥ x ∥ ( x ∈ X ) .

It follows from the first and second equation that ϕ : X → X ∗ ∗ is linear and ϕ is an isometry. Given that X is assumed to be complete, ϕ ( X ) is closed in X ∗ ∗ .

Thus, ϕ is an isometric isomorphism onto a closed subspace of X ∗ ∗ .

The members of ϕ ( x ) are exactly the linear functionals on X ∗ that are continuous with respect to its weak*-topology. Since the norm topology of X ∗ is stronger, may happen that ϕ ( X ) is a proper subspace of X ∗ ∗ .

However, there are many important spaces, such as the Lp spaces with 1 < p < ∞ , where ϕ ( X ) = X ∗ ∗ ; these are called reflexive.

It is stressed that, for X to be reflexive, the existence of some isometric isomorphism ϕ of X onto X ∗ ∗ is not enough; it is crucial that ϕ satisfies first equation in this section.

Mathematical Optimization

Let | | ⋅ | | be a norm on R n . The associated dual norm, denoted ∥ ⋅ ∥ ∗ , is defined as

| | z | | ∗ = sup { z ⊺ x | | | x | | ≤ 1 } .

(This can be shown to be a norm.) The dual norm can be interpreted as the operator norm of z ⊺ , interpreted as a 1 × n matrix, with the norm | | ⋅ | | on R n , and the absolute value on R :

| | z | | ∗ = sup { | z ⊺ x | | | | x | | ≤ 1 } .

From the definition of dual norm we have the inequality

z ⊺ x ≤ ∥ x ∥ ∥ z ∥ ∗

which holds for all x and z. The dual of the dual norm is the original norm: we have ∥ x ∥ ∗ ∗ = ∥ x ∥ for all x. (This need not hold in infinite-dimensional vector spaces.)

The dual of the Euclidean norm is the Euclidean norm, since

sup { z ⊺ x | ∥ x ∥ 2 ≤ 1 } = ∥ z ∥ 2 .

(This follows from the Cauchy-Schwarz inequality; for nonzero z, the value of x that maximises z ⊺ x over ∥ x ∥ 2 ≤ 1 is z ∥ z ∥ 2 .)

The dual of the ℓ 1 -norm is the ℓ ∞ -norm:

sup { z ⊺ x | ∥ x ∥ ∞ ≤ 1 } = ∑ i = 1 n | z i | = ∥ z ∥ 1 ,

and the dual of the ℓ ∞ -norm is the ℓ 1 -norm.

More generally, Hölder's inequality shows that the dual of the ℓ p -norm is the ℓ q -norm, where, q satisfies 1 p + 1 q = 1 , i.e., q = p p − 1 .

As another example, consider the ℓ 2 - or spectral norm on R m × n . The associated dual norm is

∥ Z ∥ 2 ∗ = sup { t r ( Z ⊺ X ) | ∥ X ∥ 2 ≤ 1 } ,

which turns out to be the sum of the singular values,

∥ Z ∥ 2 ∗ = σ 1 ( Z ) + … + σ r ( Z ) = t r ( Z ⊺ Z ) 1 2 ,

where r = r a n k Z . This norm is sometimes called the nuclear norm.

Dual norm for matrices

The Frobenius norm defined by ∥ A ∥ F = ∑ i = 1 m ∑ j = 1 n | a i j | 2 = trace ⁡ ( A ∗ A ) = ∑ i = 1 min { m , n } σ i 2 is self-dual, i.e., its dual norm is ∥ ⋅ ∥ F ′ = ∥ ⋅ ∥ F . The spectral norm, a special case of the induced norm when p = 2 , is defined by the maximum singular values of a matrix, i.e., ∥ A ∥ 2 = σ m a x ( A ) , has dual norm defined by ∥ B ∥ 2 ′ = ∑ i σ i ( B ) for any matrix B where σ i ( B ) denote the singular values.