User:Mim.cis/sandbox/Sobolev-RKHS-CA

Sobolev Smoothness and Reproducing Kernel Hilbert Space

The amount of smoothness was examined by Dupuis et.al. and Trouve using the Sobolev embedding theorem to demonstrate the necessary conditions for constraining the vector fields $v\in V$ to be in Hilbert space which is embedded in functions with at least once continuous derivative . The norm of the Hilbert space is defined via a differential operator so as to penalize derivatives in integral-square; proper choice of number of derivatives implies continuous vector fields with flows which are smooth. The Sobolev condition for 1-continuous derivatives for volumes ${\mathbb {R} }^{3}$ is that $m\geq 3/2+1$ square-integral derivatives must exist, requiring each component of the vector field to have finite Sobolev norm with 3 derivatives square-integrable $\|v_{i}\|_{H_{0}^{3}}^{2}<\infty ,i=1,2,3.$ The Hilbert space norm $\|v_{i}\|_{V}$ is constructed from a one-to-one differential operator $L=(\nabla ^{2}-c)^{p},p\geq 1.5$ to dominate the Sobolev norm

$\|v_{i}\|_{V}^{2}\doteq \int _{X}(Lv_{i})^{2}dx\geq \|v_{i}\|_{H_{0}^{3}}^{2}\doteq \sum _{\alpha :\alpha _{1}+\alpha _{2}+\alpha _{3}\leq 3}\int _{X}|{\frac {\partial ^{\alpha _{1}+\alpha _{2}+\alpha _{3}}v_{i}(x)}{\partial x_{1}^{\alpha _{1}}\partial x_{2}^{\alpha _{2}}\partial x_{3}^{\alpha _{3}}}}|^{2}dx\ .$

(Sobolev space smoothness)

The Sobolev embedding theorem dictates how much differentiation is required so that the space of vector field is continuously embedded in 1-times differentiable vector fields vanishing at infinity $V\rightarrow C_{0}^{1}({\mathbb {R} }^{3}).$ The Hilbert space of vector field $(V,\|\cdot \|_{V})$ is constructed with inner-product defined via a one-to-one differential operator $A:V\rightarrow V^{*}$, with $V^{*}$ the dual space. The dual space contains generalized vector functions or distributions $\sigma \in V^{*}$, for $X\subset {\mathbb {R} }^{3}$, then $\sigma =(\sigma _{1},\sigma _{2},\sigma _{3})\in V^{*},$ with $(\sigma |f)\doteq \int _{X}\sum _{i=1}^{3}f_{i}(x)\sigma _{i}(dx)\ ,\ for\ f\in V\ .$

We choose our Hilbert space $(V,\|\cdot \|_{V})$ with norm so that it dominates the Sobolev norm of proper order, for ${\mathbb {R} }^{3}$ then $\|v_{t}\|_{V}^{2}\doteq (Av_{t}|v_{t})\geq \|v_{t}\|_{H_{0}^{m}({\mathbb {R} }^{3})}^{2};$ finiteness of $(Av_{t}|v_{t})<\infty$ implies the Sobolev norm is finite. For d-dimensional backround space $X\subset {\mathbb {R} }^{d}$, the Sobolev norm associated to the d-components $v_{it},i=1,\dots ,d$, the necessary condition for smooth embedding with k-derivatives, $V\subset C_{0}^{k}({\mathbb {R} }^{d},{\mathbb {R} }^{d})$ must satisfy $m\geq k+d/2$

For 1-continuous derivative, the backround space $X\subset {\mathbb {R} }^{2}$, then $m\geq 2$; for $X\subset {\mathbb {R} }^{3}$, then $m\geq 2.5$.

In CA, a modelling approach used as in other branches of machine learning is to model the Hilbert space of vector fields as a reproducing kernel Hilbert space (RKHS). The construction begins by defining the squared operator $A=LL^{\dagger }$, $L^{\dagger }$ the adjoint of $L$. The Hilbert space inner-product on $\ v,w\in V$ becomes $\langle v,w\rangle _{V}\doteq (\sigma |w),\sigma \doteq Av\in V^{*}$; since$A:V\rightarrow V^{*}$, $V^{*}$ the dual space of $V$, then $Av$ can be a generalized function with the linear form definedas$(\sigma |v)\doteq \int _{X}\sum _{i}v_{i}(x)\sigma _{i}(dx)$. For proper choice of differential operators, then $(V,\|\cdot \|_{V})$ is an RKHS with kernel operator $K=A^{-1}$. The kernel smooths$K\sigma (x)_{i}\doteq \sum _{j}\int _{X}k_{ij}(x-y)\sigma _{j}(dy)$, with kernel $k$.

One operator choice for the norm is the Laplacian; in ${\mathbb {R} }^{3}$ choose, $A=(\nabla ^{2}-id)^{3}$ for which $(Av|v)<\infty$ implies 1 continuous spatial derivative for the kernel

$k(x-y)=(1+\|x-y\|)e^{-\|x-y\|},\,K(x-y)=k(x-y)Id_{3}\,$,

with $Id_{3}$ the 3x3 identity matrix. See /Sobolev-RKHS-CA for more details on the reproducing kernel Hilbert space formulation and the conditions for ${\mathbb {R} }^{3}$.

The smoothness required for Equations(Eulerian inverse,Lagrangian flow) results from the fact that the kernels $k(x,y)$ are continuously differentiable in both variables.

Our smoothness condition for smooth flows of the inverse requires control of the first derive $v_{t}\rightarrow Dv_{t}(x),x\in X$ which is true for smooth kernel $k_{ij}(x,y)$ in both variables $x,y$.

We require $v\in V$ a Hilbert space which continuously embeds in 1-times differentiable vector fields vanishing at infinity giving the group of diffeomorphisms generated from smooth flows:

$Diff_{V}\doteq \{\varphi =\phi _{1}:{\dot {\phi }}_{t}=v_{t}\circ \phi _{t},\phi _{0}=id,\int _{0}^{1}\|v_{t}\|_{V}dt<\infty \}.$

For proper choice on the operator, then $(V,\|\cdot \|_{V})$ is a reproducing kernel Hilbert space with the reproducing kernel $K=A^{-1}$, implying $KAv=v$. Therefore the operator smooths distributions $K:V^{*}\rightarrow V$ with the kernel $K(x,y)=(k_{ij}(x,y))$ and $K\sigma (x)_{i}\doteq \sum _{j}\int _{X}k_{ij}(x,y)\sigma _{j}(dy)\ .$ One example, $X\subset {\mathbb {R} }^{3}$, then d=3, p=3, $A=(\nabla ^{2}-id)^{3}$, Green's operator $K(x,y)=k(x-y)Id_{3},Id_{3}$ $k(x-y)=(1+\|x-y\|)e^{-\|x-y\|}$ .with $Id_{3}$diagonal 3x3 identity.