# Operator norm

In mathematics, the operator norm measures the "size" of certain linear operators by assigning each a real number called its operator norm. Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces.

## Introduction and definition

Given two normed vector spaces $V$  and $W$  (over the same base field, either the real numbers $\mathbb {R}$  or the complex numbers $\mathbb {C}$ ), a linear map $A:V\to W$  is continuous if and only if there exists a real number $c$  such that

$\|Av\|\leq c\|v\|\quad {\mbox{ for all }}v\in V.$

The norm on the left is the one in $W$  and the norm on the right is the one in $V$ . Intuitively, the continuous operator $A$  never increases the length of any vector by more than a factor of $c.$  Thus the image of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as bounded operators. In order to "measure the size" of $A,$  one can take the infimum of the numbers $c$  such that the above inequality holds for all $v\in V.$  This number represents the maximum scalar factor by which $\mathbb {R} ^{n}$  "lengthens" vectors. In other words, the "size" of $A$  is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of $A$  as

$\|A\|_{op}=\inf\{c\geq 0:\|Av\|\leq c\|v\|{\mbox{ for all }}v\in V\}.$

The infimum is attained as the set of all such $c$  is closed, nonempty, and bounded from below.

It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces $V$  and $W$ .

## Examples

Every real $m$ -by-$n$  matrix corresponds to a linear map from $\mathbb {R} ^{n}$  to $\mathbb {R} ^{m}.$  Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all $m$ -by-$n$  matrices of real numbers; these induced norms form a subset of matrix norms.

If we specifically choose the Euclidean norm on both $\mathbb {R} ^{n}$  and $\mathbb {R} ^{m},$  then the matrix norm given to a matrix $A$  is the square root of the largest eigenvalue of the matrix $A^{*}A$  (where $A^{*}$  denotes the conjugate transpose of $A$ ). This is equivalent to assigning the largest singular value of $A.$

Passing to a typical infinite-dimensional example, consider the sequence space $\ell ^{2},$  which is an Lp space, defined by

$l^{2}=\left\{\left(a_{n}\right)_{n\geq 1}:\;a_{n}\in \mathbb {C} ,\;\sum _{n}|a_{n}|^{2}<\infty \right\}.$

This can be viewed as an infinite-dimensional analogue of the Euclidean space $\mathbb {C} ^{n}.$  Now consider a bounded sequence $s_{\bullet }=\left(s_{n}\right)_{n=1}^{\infty }.$  The sequence $s_{\bullet }$  is an element of the space $\ell ^{\infty },$  with a norm given by

$\left\|s_{\bullet }\right\|_{\infty }=\sup _{n}\left|s_{n}\right|.$

Define an operator $T_{s}$  by pointwise multiplication:

$\left(a_{n}\right)_{n=1}^{\infty }\;{\stackrel {T_{s}}{\mapsto }}\;\ \left(s_{n}\cdot a_{n}\right)_{n=1}^{\infty }.$

The operator $T_{s}$  is bounded with operator norm

$\left\|T_{s}\right\|_{op}=\left\|s_{\bullet }\right\|_{\infty }.$

This discussion extends directly to the case where $\ell ^{2}$  is replaced by a general $L^{p}$  space with $p>1$  and $\ell ^{\infty }$  replaced by $L^{\infty }.$

## Equivalent definitions

Let $A:V\to W$  be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition $V\neq \{0\}$  then they are all equivalent:

{\begin{alignedat}{4}\|A\|_{op}&=\inf &&\{c\geq 0~&&:~\|Av\|\leq c\|v\|~&&~{\mbox{ for all }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|\leq 1~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|<1~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|\in \{0,1\}~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|=1~&&~{\mbox{ and }}~&&v\in V\}\;\;\;{\text{ this equality holds if and only if }}V\neq \{0\}\\&=\sup &&{\bigg \{}{\frac {\|Av\|}{\|v\|}}~&&:~v\neq 0~&&~{\mbox{ and }}~&&v\in V{\bigg \}}\;\;\;{\text{ this equality holds if and only if }}V\neq \{0\}.\\\end{alignedat}}

If $V=\{0\}$  then the sets in the last two rows will be empty, and consequently their supremums over the set $[-\infty ,\infty ]$  will equal $-\infty$  instead of the correct value of $0.$  If the supremum is taken over the set $[0,\infty ]$  instead, then the supremum of the empty set is $0$  and the formulas hold for any $V.$  If $A:V\to W$  is bounded then

$\|A\|_{op}=\sup \left\{\left|w^{*}(Av)\right|:\|v\|\leq 1,\left\|w^{*}\right\|\leq 1{\text{ where }}v\in V,w^{*}\in W^{*}\right\}$

and
$\|A\|_{op}=\left\|{}^{t}A\right\|_{op}$

where ${}^{t}A:W^{*}\to V^{*}$  is the transpose of $A:V\to W,$  which is the linear operator defined by $w^{*}\,\mapsto \,w^{*}\circ A.$

## Properties

The operator norm is indeed a norm on the space of all bounded operators between $V$  and $W$ . This means

$\|A\|_{op}\geq 0{\mbox{ and }}\|A\|_{op}=0{\mbox{ if and only if }}A=0,$

$\|aA\|_{op}=|a|\|A\|_{op}{\mbox{ for every scalar }}a,$

$\|A+B\|_{op}\leq \|A\|_{op}+\|B\|_{op}.$

The following inequality is an immediate consequence of the definition:

$\|Av\|\leq \|A\|_{op}\|v\|\ {\mbox{ for every }}\ v\in V.$

The operator norm is also compatible with the composition, or multiplication, of operators: if $V$ , $W$  and $X$  are three normed spaces over the same base field, and $A:V\to W$  and $B:W\to X$  are two bounded operators, then it is a sub-multiplicative norm, that is:

$\|BA\|_{op}\leq \|B\|_{op}\|A\|_{op}.$

For bounded operators on $V$ , this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.

## Table of common operator norms

Some common operator norms are easy to calculate, and others are NP-hard. Except for the NP-hard norms, all these norms can be calculated in $N^{2}$  operations (for an $N\times N$  matrix), with the exception of the $\ell _{2}-\ell _{2}$  norm (which requires $N^{3}$  operations for the exact answer, or fewer if you approximate it with the power method or Lanczos iterations).

Computability of Operator Norms
Co-domain
$\ell _{1}$  $\ell _{2}$  $\ell _{\infty }$
Domain $\ell _{1}$  Maximum $\ell _{1}$  norm of a column Maximum $\ell _{2}$  norm of a column Maximum $\ell _{\infty }$  norm of a column
$\ell _{2}$  NP-hard Maximum singular value Maximum $\ell _{2}$  norm of a row
$\ell _{\infty }$  NP-hard NP-hard Maximum $\ell _{1}$  norm of a row

The norm of the adjoint or transpose can be computed as follows. We have that for any $p,q,$  then $\|A\|_{p\rightarrow q}=\|A^{*}\|_{q'\rightarrow p'}$  where $p',q'$  are Hölder conjugate to $p,q,$  that is, $1/p+1/p'=1$  and $1/q+1/q'=1.$

## Operators on a Hilbert space

Suppose $H$  is a real or complex Hilbert space. If $A:H\to H$  is a bounded linear operator, then we have

$\|A\|_{op}=\left\|A^{*}\right\|_{op}$

and
$\left\|A^{*}A\right\|_{op}=\|A\|_{op}^{2},$

where $A^{*}$  denotes the adjoint operator of $A$  (which in Euclidean spaces with the standard inner product corresponds to the conjugate transpose of the matrix $A$ ).

In general, the spectral radius of $A$  is bounded above by the operator norm of $A$ :

$\rho (A)\leq \|A\|_{op}.$

To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator $A$  has spectrum $\{0\}.$  So $\rho (A)=0$  while $\|A\|_{op}>0.$

However, when a matrix $N$  is normal, its Jordan canonical form is diagonal (up to unitary equivalence); this is the spectral theorem. In that case it is easy to see that

$\rho (N)=\|N\|_{op}.$

This formula can sometimes be used to compute the operator norm of a given bounded operator $A$ : define the Hermitian operator $B=A^{*}A,$  determine its spectral radius, and take the square root to obtain the operator norm of $A.$

The space of bounded operators on $H,$  with the topology induced by operator norm, is not separable. For example, consider the Lp space $L^{2}[0,1],$  which is a Hilbert space. For $0  let $\Omega _{t}$  be the characteristic function of $[0,t],$  and $P_{t}$  be the multiplication operator given by $\Omega _{t},$  that is,

$P_{t}(f)=f\cdot \Omega _{t}.$

Then each $P_{t}$  is a bounded operator with operator norm 1 and

$\left\|P_{t}-P_{s}\right\|_{op}=1\quad {\mbox{ for all }}\quad t\neq s.$

But $\{P_{t}:0  is an uncountable set. This implies the space of bounded operators on $L^{2}([0,1])$  is not separable, in operator norm. One can compare this with the fact that the sequence space $\ell ^{\infty }$  is not separable.

The associative algebra of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a C*-algebra.