Fast Algorithms for Multidimensional Signals

Similar to 1-D Digital signal processing in case of the Multidimensional signal processing we have Efficient algorithms. The efficiency of an Algorithm can be evaluated by the amount of computational resources it takes to compute output or the quantity of interest. In this page, two of the very efficient algorithms for multidimensional signals are explained. For the sake of simplicity and description it is explained for 2-D Signals. However, same theory holds good for M-D signals. The exact computational savings for each algorithm is also mentioned.

Motivation and applications

In the case of digital systems, a mathematical expressions can be used to describe the input-output relationship and an algorithm can be used to implement this relationship. Similarly, algorithms can be developed to implement different transforms such as Digital filter, Fourier transform, Histogram, Image Enhancements, etc. Direct implementation of these input-output relationships and transforms is not necessarily the most efficient way to implement those.

When people began to compute such outputs from input through direct implementation, they began to look for more efficient ways. This wiki page aims at showcasing such efficient and fast algorithms for multidimensional signals and systems. A multidimensional (M-D) signal can be modeled as a function of M independent variables, where M is greater than or equal to 2. These signals may be categorized as continuous, discrete, or mixed. A continuous signal can be modeled as a function of independent variables which range over a continuum of values, example – an audio wave travelling in space, 3-D space waves measured at different times. A discrete signal, on the other hand, can be modeled as a function defined only on a set of points, such as the set of integers. An image is the simplest example of a 2-D discrete domain signal that is spatial in nature.

In the context of Fast Algorithms, consider the example below:

We need to compute A which is given by

A = αγ + αδ + βγ + βδ where α,β,γ and δ are complex variables.

To compute A, we need 4 complex multiplications and 3 complex additions. The above equation can be written in its simplified form as

A = (α + β)(γ + δ)

This form requires only 1 complex multiplication and 2 complex additions.

Thus the second way of computing A is much more efficient and fast compared to the first method of computing A. This is the motivation for the evolution of the fast algorithms in the digital signal processing Field. Consequently, many of the real-world applications make use of these efficient Algorithms for fast computations.

Problem Statement and Basics

The simplest form of representing a Linear Shift Invariant system(LSI) is through its Impulse response. The output of such LSI discrete domain system is given by the convolution of its input signal and system's impulse response. This is mathematically represented as follows:

$y\left(n_{1},n_{2}\right)=\sum _{l_{1}=-\infty }^{+\infty }\sum _{l_{2}=\infty }^{+\infty }x(l_{1},l_{2})h(n_{1}-l_{1},n_{2}-l_{2})$

where $h(n_{1},n_{2})$ is the impulse response of the system.

According to the equation above to obtain the Output value at a particular point ( say $y(0,0)$) we need to multiply several values of the input $x(l_{1},l_{2})$ and Impulse Response $h(-l_{1},-l_{2})$. Of course this is dependent on the Region of Support of the input as well as the impulse response. The key point here to be noted is that we need to perform so many complex multiplications and additions to obtain 1 output value.

Assuming a 2-D input signal is of length $M\times M$ and the system's impulse response is of length $N\times N$ we need to perform $M^{2}N^{2}$ number of multiplications to obtain all output values. The output can be computed efficiently if one can exploit some characteristics of the system.

We encounter a similar scenario when we have to compute the discrete Fourier Transforms of a signal of interest.

The Direct Calculation of the 2-D DFT is simply the evaluation of the double Sum

$X\left(k_{1},k_{2}\right)=\sum _{n_{1}=0}^{N_{1}-1}\sum _{n_{2}=0}^{N_{2}-1}x(n_{1},n_{2})W_{N_{1}}^{n_{1}k_{1}}W_{N_{2}}^{n_{2}k_{2}}$ ${\text{ for }}0\leq k_{1}\leq N_{1}-1{\text{ and }}0\leq k_{2}\leq N_{2}-1$

${\text{ where }}W_{N}\equiv {\text{exp}}\left(-j{\frac {2\pi }{N}}\right)$

The total number of complex multiplications and complex additions needed to evaluate this 2-D DFT by direct calculation is $N_{1}^{2}N_{2}^{2}$. This is a naive approach, however, we already know that an N-point 1-D DFT can be computed with far fewer than $N^{2}$ multiplications by using the Fast Fourier Transform (FFT) algorithm. As described in the next section we can develop Fast Fourier transforms for calculating 2-D or higher dimensional DFTs as well

Fast Algorithms for Multidimensional signals

Row Column Decomposition approach for the evaluation of DFT

Source:

The DFT sum $X\left(k_{1},k_{2}\right)$ in the previous equation can also be written in the following form

$X\left(k_{1},k_{2}\right)=\sum _{n_{1}=0}^{N_{1}-1}\left[\sum _{n_{2}=0}^{N_{2}-1}x(n_{1},n_{2})W_{N_{2}}^{n_{2}k_{2}}\right]W_{N_{1}}^{n_{1}k_{1}}$

Let $G\left(n_{1},k_{2}\right)$ denote the quantity inside the brackets and is given by:

$G\left(n_{1},k_{2}\right)=\sum _{n_{2}=0}^{N_{2}-1}x(n_{1},n_{2})W_{N_{2}}^{n_{2}k_{2}}$

$X\left(k_{1},k_{2}\right)=\sum _{n_{1}=0}^{N_{1}-1}G\left(n_{1},k_{2}\right)W_{N_{1}}^{n_{1}k_{2}}$

Employing this method the DFT $X$ can be computed as multiple 1-D DFTs. That is, each column of $G$ can be considered as a 1-D DFT of the corresponding column of $x$($n_{1}$ = constant). And each row of $X$ is the 1-DFT of the corresponding row of the $G$($n_{2}$ = constant). Hence we are computing the 2-D DFT by decomposing it into Row and Column DFTs.

The same principle is employed for evaluating the M-D DFT of a M - dimensional signal.

Now let's talk about the computational savings we get using this approach. It is observed that we require $N_{1}N_{2}(N_{1}+N_{2})$ complex additions and multiplications. Further, if each of these 1-D DFT is computed using a 1-D FFT, the number of complex multiplications can be further reduced to $N_{1}N_{2}{\frac {\log _{2}N_{1}N_{2}}{2}}$

Vector Radix Fast Fourier Transform

Source:

Just like the 1-D FFT, decimation in time can be achieved in the case of 2-D Signals. The 1-D DFT of a signal whose length is a power of 2, can be expressed in terms of two half-length DFTs, each of these can again be expressed as a combination of quarter length DFTs and so on.

In the case of 2-D signals we can express the $(N_{1}{\text{ x }}N_{2})$ DFT in terms of four ${\frac {N_{1}}{2}}{\text{ x }}{\frac {N_{2}}{2}}$ DFTs (assuming $N_{1}$ and $N_{2}$ are powers of 2). For the sake of simplicity let us assume that $N_{1}=N_{2}=N$. The DFT double sum can be decomposed into four separate summations, one over those samples of $x$ for which both $n_{1}$ and $n_{2}$ are even, one for which $n_{1}$ is even and $n_{2}$ is odd, one for which $n_{1}$ is odd and $n_{2}$ is even and the last one for which $n_{1}$ and $n_{2}$ are odd.

This is written as :

$X\left(k_{1},k_{2}\right)=S_{00}(k_{1},k_{2})+S_{01}(k_{1},k_{2})W_{N}^{k_{2}}+S_{10}(k_{1},k_{2})W_{N}^{k_{1}}+S_{11}(k_{1},k_{2})W_{N}^{k_{1}+k_{2}}$

where

$S_{00}(k_{1},k_{2})\equiv \sum _{m_{1}=0}^{N/2-1}\sum _{m_{2}=0}^{N/2-1}x(2m_{1},2m_{2})W_{N}^{2m_{1}k_{1}+2m_{2}k_{2}}$

$S_{01}(k_{1},k_{2})\equiv \sum _{m_{1}=0}^{N/2-1}\sum _{m_{2}=0}^{N/2-1}x(2m_{1},2m_{2}+1)W_{N}^{2m_{1}k_{1}+2m_{2}k_{2}}$

$S_{10}(k_{1},k_{2})\equiv \sum _{m_{1}=0}^{N/2-1}\sum _{m_{2}=0}^{N/2-1}x(2m_{1}+1,2m_{2})W_{N}^{2m_{1}k_{1}+2m_{2}k_{2}}$

$S_{11}(k_{1},k_{2})\equiv \sum _{m_{1}=0}^{N/2-1}\sum _{m_{2}=0}^{N/2-1}x(2m_{1}+1,2m_{2}+1)W_{N}^{2m_{1}k_{1}+2m_{2}k_{2}}$

All the arrays $S_{00}S_{01}S_{10}$ and $S_{11}$ are each periodic in $(k_{1},k_{2})$ with horizontal and vertical periods ${\frac {N}{2}}$. Using this fact and also the fact that $W_{N}^{N/2}=-1,$ we can obtain the following identities :

$X\left(k_{1},k_{2}\right)=S_{00}(k_{1},k_{2})+S_{01}(k_{1},k_{2})W_{N}^{k_{2}}+S_{10}(k_{1},k_{2})W_{N}^{k_{1}}+S_{11}(k_{1},k_{2})W_{N}^{k_{1}+k_{2}}$

$X\left(k_{1}+{\frac {N}{2}},k_{2}\right)=S_{00}(k_{1},k_{2})+S_{01}(k_{1},k_{2})W_{N}^{k_{2}}-S_{10}(k_{1},k_{2})W_{N}^{k_{1}}-S_{11}(k_{1},k_{2})W_{N}^{k_{1}+k_{2}}$

$X\left(k_{1},k_{2}+{\frac {N}{2}}\right)=S_{00}(k_{1},k_{2})-S_{01}(k_{1},k_{2})W_{N}^{k_{2}}+S_{10}(k_{1},k_{2})W_{N}^{k_{1}}-S_{11}(k_{1},k_{2})W_{N}^{k_{1}+k_{2}}$

$X\left(k_{1}+{\frac {N}{2}},k_{2}+{\frac {N}{2}}\right)=S_{00}(k_{1},k_{2})-S_{01}(k_{1},k_{2})W_{N}^{k_{2}}-S_{10}(k_{1},k_{2})W_{N}^{k_{1}}+S_{11}(k_{1},k_{2})W_{N}^{k_{1}+k_{2}}$

The above equation tell us how to compute the four DFT points $X\left(k_{1},k_{2}\right)X\left(k_{1}+{\frac {N}{2}},k_{2}\right)X\left(k_{1},k_{2}+{\frac {N}{2}}\right){\text{ and }}X\left(k_{1}+{\frac {N}{2}},k_{2}+{\frac {N}{2}}\right)$ for a particular value of $(k_{1},k_{2})$ from the four points $S_{00}(k_{1},k_{2})S_{01}(k_{1},k_{2})S_{10}(k_{1},k_{2}){\text{ and }}S_{11}(k_{1},k_{2})$. $S_{00}(k_{1},k_{2})$ can be obtained by evaluating a $\left({\frac {N}{2}}\times {\frac {N}{2}}\right)$ -point DFT (similarly other $S_{ij}$ can be obtained).

Thus we see that the $N\times N$ DFT can be expressed in terms of four ${\frac {N}{2}}\times {\frac {N}{2}}$ DFTs.

By analogy from the 1-D case, the computation depicted in the below figure is called a ${\text{butterfly}}$ or more precisely $radix-(2\times 2){\text{ butterfly}}$ .

Each butterfly requires three complex multiplications and eight complex additions for the calculation of outputs from the inputs. And to compute all the samples of $X$ from $S_{00}(k_{1},k_{2}),S_{01}(k_{1},k_{2}),S_{10}(k_{1},k_{2}){\text{ and }}S_{11}(k_{1},k_{2})$ it requires calculations of ${\frac {N^{2}}{4}}$ butterflies.

This decimation procedure is performed $\log _{2}N$ times when $N$ is a power of 2. Each stage of decimation consists of ${\frac {N^{2}}{4}}$ butterflies, and each butterfly involves three complex multiplications and eight complex additions and hence the number of complex multiplications that needs to be performed during the computation of an $(N\times N)$ -point radix $(2\times 2)$ FFT is given by

$C_{vectorRadix(2\times 2)}={\frac {3N^{2}}{4}}\log _{2}N$

See also

• Fast Fourier transform
• Multidimensional transform
• Multidimensional sampling
• Multidimensional discrete convolution
• Multidimensional filter design and Implementation