In mathematical optimization, the proximal operator is an operator associated with a proper,[note 1] lower semi-continuous convex function from a Hilbert space to , and is defined by: [1]

For any function in this class, the minimizer of the right-hand side above is unique, hence making the proximal operator well-defined. The proximal operator is used in proximal gradient methods, which is frequently used in optimization algorithms associated with non-differentiable optimization problems such as total variation denoising.

Properties

edit

The   of a proper, lower semi-continuous convex function   enjoys several useful properties for optimization.

  • Fixed points of   are minimizers of  :  .
  • Global convergence to a minimizer is defined as follows: If  , then for any initial point  , the recursion   yields convergence   as  . This convergence may be weak if   is infinite dimensional.[2]
  • The proximal operator can be seen as a generalization of the projection operator. Indeed, in the specific case where   is the 0-  indicator function   of a nonempty, closed, convex set   we have that
 
showing that the proximity operator is indeed a generalisation of the projection operator.
  • A function is firmly non-expansive if  .
  • The proximal operator of a function is related to the gradient of the Moreau envelope   of a function   by the following identity:  .
  • The proximity operator of   is characterized by inclusion  , where   is the subdifferential of  , given by
  In particular, If   is differentiable then the above equation reduces to  .

Notes

edit
  1. ^ An (extended) real-valued function f on a Hilbert space is said to be proper if it is not identically equal to  , and   is not in its image.

References

edit
  1. ^ Neal Parikh and Stephen Boyd (2013). "Proximal Algorithms" (PDF). Foundations and Trends in Optimization. 1 (3): 123–231. Retrieved 2019-01-29.
  2. ^ Bauschke, Heinz H.; Combettes, Patrick L. (2017). Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics. New York: Springer. doi:10.1007/978-3-319-48311-5. ISBN 978-3-319-48310-8.


See also

edit
edit