User:Tyler Kelso/DigitalImageProcessing


Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as the build-up of noise and distortion during processing. Since images are defined over two dimensions (perhaps more) digital image processing may be modeled in the form of multidimensional systems. The generation and development of digital image processing are mainly affected by three factors: first, the development of computers; second, the development of mathematics (especially the creation and improvement of discrete mathematics theory); third, the demand for a wide range of applications in environment, agriculture, military, industry and medical science has increased.

History

edit

Many of the techniques of digital image processing, or digital picture processing as it often was called, were developed in the 1960s, at Bell Laboratories, the Jet Propulsion Laboratory, Massachusetts Institute of Technology, University of Maryland, and a few other research facilities, with application to satellite imagery, wire-photo standards conversion, medical imaging, videophone, character recognition, and photograph enhancement.[1] The purpose of early image processing was to improve the quality of the image. It was aimed for human beings to improve the visual effect of people. In image processing, the input is a low-quality image, and the output is an image with improved quality. Common image processing include image enhancement, restoration, encoding, and compression. The first successful application was the American Jet Propulsion Laboratory (JPL). They used image processing techniques such as geometric correction, gradation transformation, noise removal, etc. on the thousands of lunar photos sent back by the Space Detector Ranger 7 in 1964, taking into account the position of the sun and the environment of the moon. The impact of the successful mapping of the moon's surface map by the computer has been a huge success. Later, more complex image processing was performed on the nearly 100,000 photos sent back by the spacecraft, so that the topographic map, color map and panoramic mosaic of the moon were obtained, which achieved extraordinary results and laid a solid foundation for human landing on the moon.[2]

The cost of processing was fairly high, however, with the computing equipment of that era. That changed in the 1970s, when digital image processing proliferated as cheaper computers and dedicated hardware became available. This led to images being processed in real-time, for some dedicated problems such as television standards conversion. As general-purpose computers became faster, they started to take over the role of dedicated hardware for all but the most specialized and computer-intensive operations. With the fast computers and signal processors available in the 2000s, digital image processing has become the most common form of image processing, and is generally used because it is not only the most versatile method, but also the cheapest.

Image sensors

edit

The basis for modern image sensors is metal-oxide-semiconductor (MOS) technology,[3] which originates from the invention of the MOSFET (MOS field-effect transistor) by Mohamed M. Atalla and Dawon Kahng at Bell Labs in 1959.[4] This led to the development of digital semiconductor image sensors, including the charge-coupled device (CCD) and later the CMOS sensor.[3]

The charge-coupled device was invented by Willard S. Boyle and George E. Smith at Bell Labs in 1969.[5] While researching MOS technology, they realized that an electric charge was the analogy of the magnetic bubble and that it could be stored on a tiny MOS capacitor. As it was fairly straightforward to fabricate a series of MOS capacitors in a row, they connected a suitable voltage to them so that the charge could be stepped along from one to the next.[3] The CCD is a semiconductor circuit that was later used in the first digital video cameras for television broadcasting.[6]

The NMOS active-pixel sensor (APS) was invented by Olympus in Japan during the mid-1980s. This was enabled by advances in MOS semiconductor device fabrication, with MOSFET scaling reaching smaller micron and then sub-micron levels.[7][8] The NMOS APS was fabricated by Tsutomu Nakamura's team at Olympus in 1985.[9] The CMOS active-pixel sensor (CMOS sensor) was later developed by Eric Fossum's team at the NASA Jet Propulsion Laboratory in 1993.[10] By 2007, sales of CMOS sensors had surpassed CCD sensors.[11]

Image compression

edit

An important development in digital image compression technology was the discrete cosine transform (DCT), a lossy compression technique first proposed by Nasir Ahmed in 1972.[12] DCT compression became the basis for JPEG, which was introduced by the Joint Photographic Experts Group in 1992.[13] JPEG compresses images down to much smaller file sizes, and has become the most widely used image file format on the Internet.[14] Its highly efficient DCT compression algorithm was largely responsible for the wide proliferation of digital images and digital photos,[15] with several billion JPEG images produced every day as of 2015.[16]

Digital signal processor (DSP)

edit

Electronic signal processing was revolutionized by the wide adoption of MOS technology in the 1970s.[17] MOS integrated circuit technology was the basis for the first single-chip microprocessors and microcontrollers in the early 1970s,[18] and then the first single-chip digital signal processor (DSP) chips in the late 1970s.[19][20] DSP chips have since been widely used in digital image processing.[19]

The discrete cosine transform (DCT) image compression algorithm has been widely implemented in DSP chips, with many companies developing DSP chips based on DCT technology. DCTs are widely used for encoding, decoding, video coding, audio coding, multiplexing, control signals, signaling, analog-to-digital conversion, formatting luminance and color differences, and color formats such as YUV444 and YUV411. DCTs are also used for encoding operations such as motion estimation, motion compensation, inter-frame prediction, quantization, perceptual weighting, entropy encoding, variable encoding, and motion vectors, and decoding operations such as the inverse operation between different color formats (YIQ, YUV and RGB) for display purposes. DCTs are also commonly used for high-definition television (HDTV) encoder/decoder chips.[21]

Medical imaging

edit

In 1972, the engineer from British company EMI Housfield invented the X-ray computed tomography device for head diagnosis, which is what is usually called CT (computer tomography). The CT nucleus method is based on the projection of the human head section and is processed by computer to reconstruct the cross-sectional image, which is called image reconstruction. In 1975, EMI successfully developed a CT device for the whole body, which obtained a clear tomographic image of various parts of the human body. In 1979, this diagnostic technique won the Nobel Prize.[2] Digital image processing technology for medical applications was inducted into the Space Foundation Space Technology Hall of Fame in 1994.[22]

Tasks

edit

Digital image processing allows the use of much more complex algorithms, and hence, can offer both more sophisticated performance at simple tasks, and the implementation of methods which would be impossible by analogue means.

In particular, digital image processing is a concrete application of, and a practical technology based on:

Some techniques which are used in digital image processing include:

Digital image transformations

edit

Filtering

edit

Digital filters are used to blur and sharpen digital images. Filtering can be performed by:

  • convolution with specifically designed kernels (filter array) in the spatial domain[23]
  • masking specific frequency regions in the frequency (Fourier) domain

The following examples show both methods:[24]

Filter type Kernel or mask Example
Original Image    
Spatial Lowpass    
Spatial Highpass    
Fourier Representation Pseudo-code:

image = checkerboard

F = Fourier Transform of image

Show Image: log(1+Absolute Value(F))

 
Fourier Lowpass    
Fourier Highpass    

Image padding in Fourier domain filtering

edit

Images are typically padded before being transformed to the Fourier space, the highpass filtered images below illustrate the consequences of different padding techniques:

Zero padded Repeated edge padded
   

Notice that the highpass filter shows extra edges when zero padded compared to the repeated edge padding.

Filtering code examples

edit

MATLAB example for spatial domain highpass filtering.

img=checkerboard(20);                           % generate checkerboard
% **************************  SPATIAL DOMAIN  ***************************
klaplace=[0 -1 0; -1 5 -1;  0 -1 0];             % Laplacian filter kernel
X=conv2(img,klaplace);                          % convolve test img with
                                                % 3x3 Laplacian kernel
figure()
imshow(X,[])                                    % show Laplacian filtered 
title('Laplacian Edge Detection')

Affine transformations

edit

Affine transformations enable basic image transformations including scale, rotate, translate, mirror and shear as is shown in the following examples:[25]

Transformation Name Affine Matrix Example
Identity    
Reflection    
Scale    
Rotate     where θ = π/6 =30°
Shear    

To apply the affine matrix to an image, the image is converted to matrix in which each entry corresponds to the pixel intensity at that location. Then each pixel's location can be represented as a vector indicating the coordinates of that pixel in the image, [x, y], where x and y are the row and column of a pixel in the image matrix. This allows the coordinate to be multiplied by an affine-transformation matrix, which gives the position that the pixel value will be copied to in the output image.

However, to allow transformations that require translation transformations, 3 dimensional homogeneous coordinates are needed. The third dimension is usually set to a non-zero constant, usually 1, so that the new coordinate is [x, y, 1]. This allows the coordinate vector to be multiplied by a 3 by 3 matrix, enabling translation shifts. So the third dimension, which is the constant 1, allows translation.

Because matrix multiplication is associative, multiple affine transformations can be combined into a single affine transformation by multiplying the matrix of each individual transformation in the order that the transformations are done. This results in a single matrix that, when applied to a point vector, gives the same result as all the individual transformations performed on the vector [x, y, 1] in sequence. Thus a sequence of affine transformation matrices can be reduced to a single affine transformation matrix.

For example, 2 dimensional coordinates only allow rotation about the origin (0, 0). But 3 dimensional homogeneous coordinates can be used to first translate any point to (0, 0), then perform the rotation, and lastly translate the origin (0, 0) back to the original point (the opposite of the first translation). These 3 affine transformations can be combined into a single matrix, thus allowing rotation around any point in the image.[26]

Applications

edit

Digital camera images

edit

Digital cameras generally include specialized digital image processing hardware – either dedicated chips or added circuitry on other chips – to convert the raw data from their image sensor into a color-corrected image in a standard image file format. When using a camera for digital photography the camera is set to take a picture or image based on the focus of the lens rather than a film camera. The technology of the first and modern day digital photography has succeeded in modeling the original process of using exposure via photographic film outdated. Along with making exposure on photographic film less practical, focused based imaging has become a modern standard within digital photography and has solidified itself as a staple of the industry.

In modern society the technology used for digital photography is not only kept within the realms of a digital camera. The same techniques that were used in digital cameras is being used in the camera phones we use as our personal pocket-sized camera. The first of the camera phones to have the capabilities as the digital cameras that came before it was the Kyocera Visual Phone VP-210. The creation of camera phones slowly has limited the gap between a portable everyday camera and a proper digital camera. Unfortunately, the camera phone is not quite as advanced as their separate digital camera counterpart. The camera phone's reasoning for being inferior to a separate digital camera is its inability to have flexible focus creates the problem of poor performance when under situations of harsh or poor lighting conditions. In addition, most camera phones are of lesser quality because of their compression needed to get them into a JPEG format. While the JPEG format does not ruin the image quality of said photos, very few camera phones offer the ability to take RAW photos that their counterpart digital cameras can.[27]

Film

edit

Westworld (1973) was the first feature film to use the digital image processing to pixellate photography to simulate an android's point of view.[28]

Analyzing the Electromagnetic Spectrum

edit

The Electromagnetic Spectrum can be analyzed through cameras via digital image processing. This analysis of the Electromagnetic Spectrum and spectrums similar to it are particularly useful in nuclear, medical and astronomical observation because of the inability of the human eye to see beyond the visible wavelength on the Electromagnetic Spectrum.[29]

Machine/Robot "Vision"

edit

Digital image processing has done a great deal of work in order to make machine/AI able to see and identify what is in front of it. Hurdle detection is a task done through image processing where a machine identifies 2 different objects and calculates the distance between the machine and the objects[30]. Line following robots are also of the work of digital image processing, using the technology to have a robot follow a line and stay on a strict path.[31]

Facial Detection

edit

Facial detection uses object class detection to be able to detect facial features and ignore everything else via digital image processing. Facial detection works similarly to Hurdle detection where it detects the sizes and locations of facial features in order to verify your facial profile. This technology is common practice within smart devices as a method to unlock the device without the use of a passcode.[32]

Digital Video Processing

edit

A video is a large collection and fast movement of pictures. Video quality can be affected by the image processing; which includes noise reduction, detail enhancement, motion detection, frame rate conversion, aspect ratio conversion and color space conversion.[30]

Underwater Image Restoration & Enhancement

edit

When light enters water it exponentially attenuates with the depth of the water level; therefore the visibility distance is affected and limited. Digital image processing can be used in the forms of restoration and enhancement of the underwater images to make them more clear and thus easier to research.[29]

Impact of Digital Cameras

edit

The widespread availability of digital cameras has created a much greater opportunity for a large group of people to be introduced into the world of photography. The ability to have a larger group of people enter the world of photography opened up opportunities for business to have plenty of options when it came to photographers. In addition to having an increasing amount of people interested in photography, the age of digital cameras and digital photography transformed the industry of photography by completely eliminating the delay and costs of the previous photography methods. The ease of viewing and transferring digital photo medias allowed photos to be edited, transferred and managed from home computers rather than with special equipment. The creation of camera phones however, has had one of the largest impacts in digital camera photography. Camera phones have allowed the user to take high quality and compressed photos from a device slim enough to fit in their pocket. Additionally, camera phones allow the user to instantly upload their digital photos to the internet where the photos can be preserved regardless of the camera being destroyed or deleted. The ability to have a digital copy of photos has completely revolutionized the way the average person approaches photography.[33]

Fatigue detection and monitoring technologies

edit

There were significant advancements in fatigue monitoring technology the past decade. These innovative technology solutions are now commercially available and offer real safety benefits to drivers, operators and other shift workers across all industries.[citation needed]

Software developers, engineers and scientists develop fatigue detection software using various physiological cues to determine the state of fatigue or drowsiness. The measurement of brain activity (electroencephalogram) is widely accepted as the standard in fatigue monitoring. Other technology used to determine fatigue related impairment include behavioral symptom measurements such as; eye behavior, gaze direction, micro-corrections in steering and throttle use as well as heart rate variability.[citation needed]

See also

edit

References

edit
  1. ^ Azriel Rosenfeld, Picture Processing by Computer, New York: Academic Press, 1969
  2. ^ a b Gonzalez, Rafael C. (2008). Digital image processing. Woods, Richard E. (Richard Eugene), 1954- (3rd ed.). Upper Saddle River, N.J.: Prentice Hall. pp. 23–28. ISBN 9780131687288. OCLC 137312858.
  3. ^ a b c Williams, J. B. (2017). The Electronics Revolution: Inventing the Future. Springer. pp. 245–8. ISBN 9783319490885.
  4. ^ "1960: Metal Oxide Semiconductor (MOS) Transistor Demonstrated". The Silicon Engine. Computer History Museum. Archived from the original on 3 October 2019. Retrieved 31 August 2019.
  5. ^ James R. Janesick (2001). Scientific charge-coupled devices. SPIE Press. pp. 3–4. ISBN 978-0-8194-3698-6.
  6. ^ Boyle, William S; Smith, George E. (1970). "Charge Coupled Semiconductor Devices". Bell Syst. Tech. J. 49 (4): 587–593. doi:10.1002/j.1538-7305.1970.tb01790.x.
  7. ^ Fossum, Eric R. (12 July 1993). "Active pixel sensors: Are CCDS dinosaurs?". In Blouke, Morley M. (ed.). Charge-Coupled Devices and Solid State Optical Sensors III. Proceedings of the SPIE. Vol. 1900. pp. 2–14. Bibcode:1993SPIE.1900....2F. CiteSeerX 10.1.1.408.6558. doi:10.1117/12.148585. S2CID 10556755.
  8. ^ Fossum, Eric R. (2007). "Active Pixel Sensors" (Document). {{cite document}}: Cite document requires |publisher= (help); Unknown parameter |s2cid= ignored (help)
  9. ^ Matsumoto, Kazuya; et al. (1985). "A new MOS phototransistor operating in a non-destructive readout mode". Japanese Journal of Applied Physics. 24 (5A): L323. Bibcode:1985JaJAP..24L.323M. doi:10.1143/JJAP.24.L323.
  10. ^ Fossum, Eric R.; Hondongwa, D. B. (2014). "A Review of the Pinned Photodiode for CCD and CMOS Image Sensors". IEEE Journal of the Electron Devices Society. 2 (3): 33–43. doi:10.1109/JEDS.2014.2306412.
  11. ^ "CMOS Image Sensor Sales Stay on Record-Breaking Pace". IC Insights. 8 May 2018. Archived from the original on 21 June 2019. Retrieved 6 October 2019.
  12. ^ Ahmed, Nasir (January 1991). "How I Came Up With the Discrete Cosine Transform". Digital Signal Processing. 1 (1): 4–5. doi:10.1016/1051-2004(91)90086-Z. Archived from the original on 10 June 2016. Retrieved 10 October 2019.
  13. ^ "T.81 – DIGITAL COMPRESSION AND CODING OF CONTINUOUS-TONE STILL IMAGES – REQUIREMENTS AND GUIDELINES" (PDF). CCITT. September 1992. Archived (PDF) from the original on 17 July 2019. Retrieved 12 July 2019.
  14. ^ "The JPEG image format explained". BT.com. BT Group. 31 May 2018. Archived from the original on 5 August 2019. Retrieved 5 August 2019.
  15. ^ "What Is a JPEG? The Invisible Object You See Every Day". The Atlantic. 24 September 2013. Archived from the original on 9 October 2019. Retrieved 13 September 2019.
  16. ^ Baraniuk, Chris (15 October 2015). "Copy protections could come to JPEGs". BBC News. BBC. Archived from the original on 9 October 2019. Retrieved 13 September 2019.
  17. ^ Grant, Duncan Andrew; Gowar, John (1989). Power MOSFETS: theory and applications. Wiley. p. 1. ISBN 9780471828679. The metal-oxide-semiconductor field-effect transistor (MOSFET) is the most commonly used active device in the very large-scale integration of digital integrated circuits (VLSI). During the 1970s these components revolutionized electronic signal processing, control systems and computers.
  18. ^ Shirriff, Ken (30 August 2016). "The Surprising Story of the First Microprocessors". IEEE Spectrum. 53 (9). Institute of Electrical and Electronics Engineers: 48–54. doi:10.1109/MSPEC.2016.7551353. S2CID 32003640. Archived from the original on 13 October 2019. Retrieved 13 October 2019.
  19. ^ a b "1979: Single Chip Digital Signal Processor Introduced". The Silicon Engine. Computer History Museum. Archived from the original on 3 October 2019. Retrieved 14 October 2019.
  20. ^ Taranovich, Steve (27 August 2012). "30 years of DSP: From a child's toy to 4G and beyond". EDN. Archived from the original on 14 October 2019. Retrieved 14 October 2019.
  21. ^ Stanković, Radomir S.; Astola, Jaakko T. (2012). "Reminiscences of the Early Work in DCT: Interview with K.R. Rao" (PDF). Reprints from the Early Days of Information Sciences. 60. Archived (PDF) from the original on 13 October 2019. Retrieved 13 October 2019.
  22. ^ "Space Technology Hall of Fame:Inducted Technologies/1994". Space Foundation. 1994. Archived from the original on 4 July 2011. Retrieved 7 January 2010.
  23. ^ Zhang, M. Z.; Livingston, A. R.; Asari, V. K. (2008). "A High Performance Architecture for Implementation of 2-D Convolution with Quadrant Symmetric Kernels". International Journal of Computers and Applications. 30 (4): 298–308. doi:10.1080/1206212x.2008.11441909. S2CID 57289814.
  24. ^ Gonzalez, Rafael (2008). Digital Image Processing, 3rd. Pearson Hall. ISBN 9780131687288.
  25. ^ Gonzalez, Rafael (2008). Digital Image Processing, 3rd. Pearson Hall. ISBN 9780131687288.
  26. ^ House, Keyser (6 December 2016). Affine Transformations (PDF). Foundations of Physically Based Modeling & Animation. A K Peters/CRC Press. ISBN 9781482234602. Archived (PDF) from the original on 30 August 2017. Retrieved 26 March 2019. {{cite book}}: |website= ignored (help)
  27. ^ "Digital photography", Wikipedia, 2 December 2020, retrieved 15 December 2020
  28. ^ A Brief, Early History of Computer Graphics in Film Archived 17 July 2012 at the Wayback Machine, Larry Yaeger, 16 August 2002 (last update), retrieved 24 March 2010
  29. ^ a b Demirel, Hasan. "Digital Image Processing" (PDF). Digital Image Processing. EE-583.
  30. ^ a b Ravikumar, R. (December 2018). "Digital Image Processing-A Quick Review" (PDF). International Journal of Intelligent Computing and Technology. Vol.2: 16–24 – via IJICT. {{cite journal}}: |volume= has extra text (help)
  31. ^ Sarwade, J.; Shetty, S.; Bhavsar, A.; Mergu, M.; Talekar, A. (2019-03). "Line Following Robot Using Image Processing". 2019 3rd International Conference on Computing Methodologies and Communication (ICCMC): 1174–1179. doi:10.1109/ICCMC.2019.8819826. {{cite journal}}: Check date values in: |date= (help)
  32. ^ Padmappriya, S. "Digital Image Processing Real Time Applications" (PDF). International Journal of Engineering Science Invention: 46–51 – via IJESI.
  33. ^ "Digital photography", Wikipedia, 2 December 2020, retrieved 15 December 2020

Further reading

edit
  • R. Fisher; K Dawson-Howe; A. Fitzgibbon; C. Robertson; E. Trucco (2005). Dictionary of Computer Vision and Image Processing. John Wiley. ISBN 978-0-470-01526-1.
  • Rafael C. Gonzalez; Richard E. Woods; Steven L. Eddins (2004). Digital Image Processing using MATLAB. Pearson Education. ISBN 978-81-7758-898-9.
  • Tyagi Vipin (2018). Understanding Digital Image Processing. Taylor and Francis CRC Press. ISBN 978-11-3856-6842.
  • Milan Sonka; Vaclav Hlavac; Roger Boyle (1999). Image Processing, Analysis, and Machine Vision. PWS Publishing. ISBN 978-0-534-95393-5.
  • Rafael C. Gonzalez (2008). Digital Image Processing. Prentice Hall. ISBN 9780131687288
edit