Operator norm

In mathematics, the operator norm measures the "size" of certain linear operators by assigning each a real number called its operator norm. Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces.

Introduction and definitionEdit

Given two normed vector spaces   and   (over the same base field, either the real numbers   or the complex numbers  ), a linear map   is continuous if and only if there exists a real number   such that[1]


The norm on the left is the one in   and the norm on the right is the one in  . Intuitively, the continuous operator   never increases the length of any vector by more than a factor of   Thus the image of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as bounded operators. In order to "measure the size" of   one can take the infimum of the numbers   such that the above inequality holds for all   This number represents the maximum scalar factor by which   "lengthens" vectors. In other words, the "size" of   is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of   as


The infimum is attained as the set of all such   is closed, nonempty, and bounded from below.[2]

It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces   and  .


Every real  -by-  matrix corresponds to a linear map from   to   Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all  -by-  matrices of real numbers; these induced norms form a subset of matrix norms.

If we specifically choose the Euclidean norm on both   and   then the matrix norm given to a matrix   is the square root of the largest eigenvalue of the matrix   (where   denotes the conjugate transpose of  ).[3] This is equivalent to assigning the largest singular value of  

Passing to a typical infinite-dimensional example, consider the sequence space   which is an Lp space, defined by


This can be viewed as an infinite-dimensional analogue of the Euclidean space   Now consider a bounded sequence   The sequence   is an element of the space   with a norm given by


Define an operator   by pointwise multiplication:


The operator   is bounded with operator norm


This discussion extends directly to the case where   is replaced by a general   space with   and   replaced by  

Equivalent definitionsEdit

Let   be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition   then they are all equivalent:


If   then the sets in the last two rows will be empty, and consequently their supremums over the set   will equal   instead of the correct value of   If the supremum is taken over the set   instead, then the supremum of the empty set is   and the formulas hold for any   If   is bounded then[4]

where   is the transpose of   which is the linear operator defined by  


The operator norm is indeed a norm on the space of all bounded operators between   and  . This means


The following inequality is an immediate consequence of the definition:


The operator norm is also compatible with the composition, or multiplication, of operators: if  ,   and   are three normed spaces over the same base field, and   and   are two bounded operators, then it is a sub-multiplicative norm, that is:


For bounded operators on  , this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.

Table of common operator normsEdit

Some common operator norms are easy to calculate, and others are NP-hard. Except for the NP-hard norms, all these norms can be calculated in   operations (for an   matrix), with the exception of the   norm (which requires   operations for the exact answer, or fewer if you approximate it with the power method or Lanczos iterations).

Computability of Operator Norms[5]
Domain   Maximum   norm of a column Maximum   norm of a column Maximum   norm of a column
  NP-hard Maximum singular value Maximum   norm of a row
  NP-hard NP-hard Maximum   norm of a row

The norm of the adjoint or transpose can be computed as follows. We have that for any   then   where   are Hölder conjugate to   that is,   and  

Operators on a Hilbert spaceEdit

Suppose   is a real or complex Hilbert space. If   is a bounded linear operator, then we have

where   denotes the adjoint operator of   (which in Euclidean spaces with the standard inner product corresponds to the conjugate transpose of the matrix  ).

In general, the spectral radius of   is bounded above by the operator norm of  :


To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator   has spectrum   So   while  

However, when a matrix   is normal, its Jordan canonical form is diagonal (up to unitary equivalence); this is the spectral theorem. In that case it is easy to see that


This formula can sometimes be used to compute the operator norm of a given bounded operator  : define the Hermitian operator   determine its spectral radius, and take the square root to obtain the operator norm of  

The space of bounded operators on   with the topology induced by operator norm, is not separable. For example, consider the Lp space   which is a Hilbert space. For   let   be the characteristic function of   and   be the multiplication operator given by   that is,


Then each   is a bounded operator with operator norm 1 and


But   is an uncountable set. This implies the space of bounded operators on   is not separable, in operator norm. One can compare this with the fact that the sequence space   is not separable.

The associative algebra of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a C*-algebra.

See alsoEdit


  1. ^ Kreyszig, Erwin (1978), Introductory functional analysis with applications, John Wiley & Sons, p. 97, ISBN 9971-51-381-1
  2. ^ See e.g. Lemma 6.2 of Aliprantis & Border (2007).
  3. ^ Weisstein, Eric W. "Operator Norm". mathworld.wolfram.com. Retrieved 2020-03-14.
  4. ^ a b Rudin 1991, pp. 92–115.
  5. ^ section 4.3.1, Joel Tropp's PhD thesis, [1]