Open main menu
Visual comparison of convolution, cross-correlation and autocorrelation. For the operations involving function f, and assuming the height of f is 1.0, the value of the result at 5 different points is indicated by the shaded area below each point. Also, the vertical symmetry of f is the reason and are identical in this example.

In signal processing, cross-correlation is a measure of similarity of two series as a function of the displacement of one relative to the other. This is also known as a sliding dot product or sliding inner-product. It is commonly used for searching a long signal for a shorter, known feature. It has applications in pattern recognition, single particle analysis, electron tomography, averaging, cryptanalysis, and neurophysiology. The cross-correlation is similar in nature to the convolution of two functions. In an autocorrelation, which is the cross-correlation of a signal with itself, there will always be a peak at a lag of zero, and its size will be the signal energy.

In probability and statistics, the term cross-correlations refers to the correlations between the entries of two random vectors and , while the correlations of a random vector are the correlations between the entries of itself, those forming the correlation matrix of . If each of and is a scalar random variable which is realized repeatedly in a time series, then the correlations of the various temporal instances of are known as autocorrelations of , and the cross-correlations of with across time are temporal cross-correlations. In probability and statistics, the definition of correlation always includes a standardising factor in such a way that correlations have values between −1 and +1.

If and are two independent random variables with probability density functions and , respectively, then the probability density of the difference is formally given by the cross-correlation (in the signal-processing sense) ; however, this terminology is not used in probability and statistics. In contrast, the convolution (equivalent to the cross-correlation of and ) gives the probability density function of the sum .

Cross-correlation of deterministic signalsEdit

For continuous functions   and  , the cross-correlation is defined as:[1][2][3]

 

 

 

 

 

(Eq.1)

which is equivalent to

 

where   denotes the complex conjugate of  , and   is the displacement, also known as lag (a feature in   at   occurs in   at  ).

Similarly, for discrete functions, the cross-correlation is defined as:[4][5]

 

 

 

 

 

(Eq.2)

which is equivalent to

 .

ExplanationEdit

As an example, consider two real valued functions   and   differing only by an unknown shift along the x-axis. One can use the cross-correlation to find how much   must be shifted along the x-axis to make it identical to  . The formula essentially slides the   function along the x-axis, calculating the integral of their product at each position. When the functions match, the value of   is maximized. This is because when peaks (positive areas) are aligned, they make a large contribution to the integral. Similarly, when troughs (negative areas) align, they also make a positive contribution to the integral because the product of two negative numbers is positive.

 
Animation displaying visually how cross correlation is calculated

With complex-valued functions   and  , taking the conjugate of   ensures that aligned peaks (or aligned troughs) with imaginary components will contribute positively to the integral.

In econometrics, lagged cross-correlation is sometimes referred to as cross-autocorrelation.[6]:p. 74

PropertiesEdit

  • The cross-correlation of functions   and   is equivalent to the convolution (denoted by  ) of   and  . That is:
     
  •  
  • If   is a Hermitian function, then  
  • If both   and   are Hermitian, then  .
  •  .
  • Analogous to the convolution theorem, the cross-correlation satisfies
     
where   denotes the Fourier transform, and an   again indicates the complex conjugate of  , since  . Coupled with fast Fourier transform algorithms, this property is often exploited for the efficient numerical computation of cross-correlations [7] (see circular cross-correlation).
  • The cross-correlation is related to the spectral density (see Wiener–Khinchin theorem).
  • The cross-correlation of a convolution of   and   with a function   is the convolution of the cross-correlation of   and   with the kernel  :
     .

Definition for periodic signalsEdit

If   and   are both continuous periodic functions of period  , the integration from   to   is replaced by integration over any interval   of length  :

 

which is equivalent to

 

Cross-correlation of random vectorsEdit

DefinitionEdit

For random vectors   and  , each containing random elements whose expected value and variance exist, the cross-correlation matrix of   and   is defined by[8]:p.337

 

 

 

 

 

(Eq.3)

and has dimensions  . Written component-wise:

 

The random vectors   and   need not have the same dimension, and either might be a scalar value.

ExampleEdit

For example, if   and   are random vectors, then   is a   matrix whose  -th entry is  .

Definition for complex random vectorsEdit

If   and   are complex random vectors, each containing random variables whose expected value and variance exist, the cross-correlation matrix of   and   is defined by

 

where   denotes Hermitian transposition.

Cross-correlation of stochastic processesEdit

In time series analysis and statistics, the cross-correlation of a pair of random process is the correlation between values of the processes at different times, as a function of the two times. Let   be a pair of random processes, and   be any point in time (  may be an integer for a discrete-time process or a real number for a continuous-time process). Then   is the value (or realization) produced by a given run of the process at time  .

Cross-correlation functionEdit

Suppose that the process has means   and   and variances   and   at time  , for each  . Then the definition of the cross-correlation between times   and   is[8]:p.392

 

 

 

 

 

(Eq.4)

where   is the expected value operator. Note that this expression may be not defined.

Cross-covariance functionEdit

Subtracting the mean before multiplication yields the cross-covariance between times   and  :[8]:p.392

 

 

 

 

 

(Eq.5)

Note that this expression is not well-defined for all-time series or processes, because the mean may not exist, or the variance may not exist.

Definition for wide-sense stationary stochastic processEdit

Let   represent a pair of stochastic processes that are jointly wide-sense stationary. Then the Cross-covariance function and the cross-correlation function are given as follows.

Cross-correlation functionEdit

 

 

 

 

 

(Eq.6)

or equivalently

 

Cross-covariance functionEdit

 

 

 

 

 

(Eq.7)

or equivalently

 

where   and   are the mean and standard deviation of the process  , which are constant over time due to stationarity; and similarly for  , respectively.   indicates the expected value. That the cross-covariance and cross-correlation are independent of   is precisely the additional information (beyond being individually wide-sense stationary) conveyed by the requirement that   are jointly wide-sense stationary.

The cross-correlation of a pair of jointly wide sense stationary stochastic processes can be estimated by averaging the product of samples measured from one process and samples measured from the other (and its time shifts). The samples included in the average can be an arbitrary subset of all the samples in the signal (e.g., samples within a finite time window or a sub-sampling[which?] of one of the signals). For a large number of samples, the average converges to the true cross-correlation.

NormalizationEdit

It is common practice in some disciplines (e.g. statistics and time series analysis) to normalize the cross-correlation function to get a time-dependent Pearson correlation coefficient. However, in other disciplines (e.g. engineering) the normalization is usually dropped and the terms "cross-correlation" and "cross-covariance" are used interchangeably.

The definition of the normalized cross-correlation of a stochastic process is

 .

If the function   is well-defined, its value must lie in the range  , with 1 indicating perfect correlation and −1 indicating perfect anti-correlation.

For jointly wide-sense stationary stochastic processes, the definition is

 .

The normalization is important both because the interpretation of the autocorrelation as a correlation provides a scale-free measure of the strength of statistical dependence, and because the normalization has an effect on the statistical properties of the estimated autocorrelations.

PropertiesEdit

Symmetry propertyEdit

For jointly wide-sense stationary stochastic processes, the cross-correlation function has the following symmetry property:[9]:p.173

 

Respectively for jointly WSS processes:

 

Time delay analysisEdit

Cross-correlations are useful for determining the time delay between two signals, e.g., for determining time delays for the propagation of acoustic signals across a microphone array.[10][11][clarification needed] After calculating the cross-correlation between the two signals, the maximum (or minimum if the signals are negatively correlated) of the cross-correlation function indicates the point in time where the signals are best aligned; i.e., the time delay between the two signals is determined by the argument of the maximum, or arg max of the cross-correlation, as in

 

Terminology in image processingEdit

Zero-normalized cross-correlation (ZNCC)Edit

For image-processing applications in which the brightness of the image and template can vary due to lighting and exposure conditions, the images can be first normalized. This is typically done at every step by subtracting the mean and dividing by the standard deviation. That is, the cross-correlation of a template,   with a subimage   is

 .

where   is the number of pixels in   and  ,   is the average of   and   is standard deviation of  .

In functional analysis terms, this can be thought of as the dot product of two normalized vectors. That is, if

 

and

 

then the above sum is equal to

 

where   is the inner product and   is the L² norm.

Thus, if   and   are real matrices, their normalized cross-correlation equals the cosine of the angle between the unit vectors   and  , being thus   if and only if   equals   multiplied by a positive scalar.

Normalized correlation is one of the methods used for template matching, a process used for finding incidences of a pattern or object within an image. It is also the 2-dimensional version of Pearson product-moment correlation coefficient.

Normalized cross-correlation (NCC)Edit

NCC is similar to ZNCC with the only difference of not subtracting the local mean value of intensities:

 

Nonlinear systemsEdit

Caution must be applied when using cross correlation for nonlinear systems. In certain circumstances, which depend on the properties of the input, cross correlation between the input and output of a system with nonlinear dynamics can be completely blind to certain nonlinear effects.[12] This problem arises because some quadratic moments can equal zero and this can incorrectly suggest that there is little "correlation" (in the sense of statistical dependence) between two signals, when in fact the two signals are strongly related by nonlinear dynamics.

See alsoEdit

ReferencesEdit

  1. ^ Bracewell, R. "Pentagram Notation for Cross Correlation." The Fourier Transform and Its Applications. New York: McGraw-Hill, pp. 46 and 243, 1965.
  2. ^ Papoulis, A. The Fourier Integral and Its Applications. New York: McGraw-Hill, pp. 244–245 and 252-253, 1962.
  3. ^ Weisstein, Eric W. "Cross-Correlation." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/Cross-Correlation.html
  4. ^ Rabiner, L.R.; Schafer, R.W. (1978). Digital Processing of Speech Signals. Signal Processing Series. Upper Saddle River, NJ: Prentice Hall. pp. 147–148. ISBN 0132136031.
  5. ^ Rabiner, Lawrence R.; Gold, Bernard (1975). Theory and Application of Digital Signal Processing. Englewood Cliffs, NJ: Prentice-Hall. p. 401. ISBN 0139141014.
  6. ^ Campbell; Lo; MacKinlay (1996). The Econometrics of Financial Markets. NJ: Princeton University Press. ISBN 0691043019.
  7. ^ Kapinchev, Konstantin; Bradu, Adrian; Barnes, Frederick; Podoleanu, Adrian (2015). "GPU implementation of cross-correlation for image generation in real time". 2015 9th International Conference on Signal Processing and Communication Systems (ICSPCS). pp. 1–6. doi:10.1109/ICSPCS.2015.7391783. ISBN 978-1-4673-8118-5.
  8. ^ a b c Gubner, John A. (2006). Probability and Random Processes for Electrical and Computer Engineers. Cambridge University Press. ISBN 978-0-521-86470-1.
  9. ^ Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3
  10. ^ Rhudy, Matthew; Brian Bucci; Jeffrey Vipperman; Jeffrey Allanach; Bruce Abraham (November 2009). "Microphone Array Analysis Methods Using Cross-Correlations". Proceedings of 2009 ASME International Mechanical Engineering Congress, Lake Buena Vista, FL: 281–288. doi:10.1115/IMECE2009-10798. ISBN 978-0-7918-4388-8.
  11. ^ Rhudy, Matthew (November 2009). "Real Time Implementation of a Military Impulse Classifier". University of Pittsburgh, Master's Thesis. Cite journal requires |journal= (help)
  12. ^ Billings, S. A. (2013). Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains. Wiley. ISBN 978-1-118-53556-1.

Further readingEdit

  • Tahmasebi, Pejman; Hezarkhani, Ardeshir; Sahimi, Muhammad (2012). "Multiple-point geostatistical modeling based on the cross-correlation functions". Computational Geosciences. 16 (3): 779–797. doi:10.1007/s10596-012-9287-1.

External linksEdit