Real-time computer graphics
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages)(Learn how and when to remove this template message)
Real-time computer graphics or real-time rendering is the sub-field of computer graphics focused on producing and analyzing images in real time. The term can refer to anything from rendering an application's graphical user interface (GUI) to real-time image analysis but is most often used in reference to interactive 3D computer graphics, typically using a graphics processing unit (GPU). One example of this concept is a video game that rapidly renders changing 3D environments to produce an illusion of motion.
Computers have been capable of generating 2D images such as simple lines, images, and polygons in real time since their invention. However, quickly rendering detailed 3D objects has always been a daunting task for traditional Von Neumann architecture-based systems. An early workaround to this problem was the use of sprites, 2D images that could imitate 3D graphics.
Different techniques for rendering now exist, such as ray-tracing and rasterization. Using these techniques and advanced hardware, computers can now render images quickly enough to induce an illusion of motion while simultaneously accepting user input. This means that the user can respond to rendered images in real time, producing an interactive experience.
Principles of real-time 3D computer graphicsEdit
The goal of computer graphics is to generate computer-generated images, or frames, using certain desired metrics. One such metric is the number of frames generated in a given second. Real-time computer graphics systems differ from traditional offline (i.e., non-real-time) rendering systems in that non-real-time graphics typically rely on ray tracing. In this process, millions or even billions of rays are traced from the camera to the world for detailed rendering—this expensive operation can take hours or days to render a single frame.
Real-time graphics systems, by contrast, must usually render each image in less than 1/30th of a second. Ray tracing is far too slow for these systems; instead, they employ the technique of z-buffer triangle rasterization. In this technique, every object is decomposed into individual primitives usually triangles. Each triangle gets positioned, rotated, and scaled on the screen, and special hardware (or a software emulator) called the rasterizer generates pixels inside each of these triangles. These triangles are then decomposed into atomic units called fragments that are suitable for displaying on a display screen. The fragments are drawn on the screen using a color that is often computed in several steps. For example, a texture can be used to "paint" a triangle based on a stored image, and then a technique called shadow mapping can alter that triangle's colors based on line-of-sight to light sources.
Video game graphicsEdit
Real-time graphics aims to optimize image quality subject to time constraints and hardware limitations. Significant recent advancements in hardware, particularly graphics processing units, have increased the image quality that real-time graphics can produce. Modern GPUs are capable of handling millions of triangles per frame, and current DirectX 11/OpenGL 4.x class hardware is capable of generating complex effects, such as shadow volumes, motion blurring, and real-time triangle generation, in real-time. The advancement of real-time graphics is evidenced in the progressive improvements between actual gameplay graphics and the pre-rendered cutscenes that are typically found in modern video games. As hardware and game engines have advanced, cutscenes are now typically rendered in real-time—and may be interactive. Although the gap in quality between real-time graphics and traditional off-line graphics is narrowing, offline rendering remains much more accurate.
Real-time graphics are typically employed when interactivity (e.g., player feedback) is crucial. When real-time graphics are used in films, the director has complete control of what has to be drawn on each frame, which can sometimes take weeks or even years of decision-making. Teams of people are typically involved in the making of these decisions.
When real-time computer graphics are used, the user typically uses an input device to influence what is about to be drawn on the display. For example, when the user wants to move a character on the screen, the system updates the character's position before drawing the next frame. Usually, the display's response-time is far slower than the input device—this is justified by the immense difference between the very fast response time of a human being's motion and the very slow perspective speed of the human visual system. This difference has other effects too: because input devices must be very fast to keep up with human motion response, advancements in input devices (e.g., the current Wii remote) typically take much longer to achieve than comparable advancements in display devices.
Another important factor controlling real-time computer graphics is the combination of physics and animation. These techniques largely dictate what is to be drawn on the screen—or more precisely, where to draw certain objects on the screen. These techniques help realistically imitate behavior seen in the real world (the temporal dimension, not the spatial dimensions), adding to the computer graphics' degree of realism.
Lastly, real-time previewing within a graphics software, especially when adjusting lighting effects, can increase work speed. Some parameter adjustments in fractal generating software may be made while viewing changes to the image in real time.
Graphics rendering pipelineEdit
The graphics rendering pipeline ("rendering pipeline", or simply "pipeline") is the foundation of real-time graphics. Its main function is to render a two-dimensional image in relation to a virtual camera, three-dimensional objects (an object that has width, length, and depth), light sources, lighting models, textures, and more.
The architecture of the real-time rendering pipeline can be divided into three conceptual stages: Application, Geometry, and Rasterization. This structure is the core of real-time computer graphics applications.
The application stage of a real-time graphics application is responsible for generating "scenes", or 3D settings that are to be drawn to a 2D display. This stage is implemented in software that developers carefully optimize for performance. This stage may perform processing such as collision detection, speed-up techniques, animation, and force feedback, in addition to handling user input.
Collision detection is an example of an operation that would be performed in the application stage. Collision detection uses algorithms to detect and respond to collisions between virtual objects. For example, the application may calculate new positions for the colliding objects and provide feedback via a force feedback device such as a vibrating game controller.
The application stage is also responsible for preparing graphics data for the next pipeline stage. This includes performing texture animation, animation of 3D models, animation via transforms, and geometry morphing. Finally, it produces primitives (points, lines, and triangles) based on scene information and feeds those primitives into the geometry stage of the pipeline.
The geometry stage manipulates polygons and vertices to compute what to draw, how to draw it, and where to draw it. Usually, these operations are performed by specialized hardware or graphics processing units. Variations across graphics hardware mean that the "geometry stage" may actually be implemented as several consecutive stages.
Model and view transformationEdit
Before the final model is shown on the output device, the model is transformed onto several different spaces or coordinate systems. Transformations move and manipulate objects by altering their vertices. Transformation is the general term for the four specific ways that allow one to manipulate the shape or position of a point, line, or shape.
In order to give the model a more realistic appearance, one or more light sources are usually equipped during the transforming of the model. However, this stage cannot be reached without completing the 3D scene that is being transformed into the view space. The view space is where the camera is placed at the origin and is aimed in a way that the camera is looking in the direction of the negative z-axis (where the y-axis is pointing upwards and the x-axis is pointing to the right).
Projection is a transformation used to represent a 3D model in a 2D space. The two main types of projection are orthographic projection (also called parallel) and perspective projection. The main characteristic of orthographic projection is that parallel lines remain parallel after the transformation. Perspective projection utilizes the concept that if the distance between a camera and model increases, the model will appear smaller than before. Essentially, perspective projection mimics the way human sight operates.
Clipping is the process of removing primitives that are outside of the view box in order to continue on to the rasterizer stage. Once the primitives that are outside of the view box are removed, the primitives that are still inside of the view box will be drawn into new triangles that will proceed on to the next stage.
The purpose of screen mapping is to find out the coordinates of the primitives that were determined to be on the inside of the view box during the clipping stage.
Once all of the necessary steps are completed from the two previous stages, all the elements are ready to enter the rasterizer stage. This includes the lines that have been drawn and the models that have been transformed. During the rasterizer stage, color is applied and all of the elements previously stated are turned into pixels or picture elements.
The beginnings of 3D computer graphicsEdit
Computer animation has been around since the 1940s and 1950s, but it was not until the 1970s and 1980s that 3D techniques were implemented [see the History of computer animation]. The first step towards 3D graphics was taken in 1972 by Edwin Catmull and Fred Parke. Both were students at the University of Utah at the time. Their implementation of 3D technology featured a computer-generated hand and face that was created using wire-frame imagery. Up until 1975, wire-frame imagery was the only technology used to create 3D images on computers and in films. Today, some 3D graphics have advanced to the point where animated humans look almost entirely realistic. Eventually, humans may not be able to tell the difference between real human footage and animated human footage. One movie that got close to fooling the human eye is Beowulf. The film was created using 3D motion capture technology.
- Spraul, V. Anton (2013). How Software Works: The Magic Behind Encryption, CGI, Search Engines and Other Everyday Technologies. No Starch Press. p. 86. ISBN 1593276664. Retrieved 24 September 2017.
- Wolf, Mark J. P. (2008). The Video Game Explosion: A History from PONG to Playstation and Beyond. ABC-CLIO. p. 86. ISBN 9780313338687. Retrieved 24 September 2017.
- Birn, Jeremy (2013). Digital Lighting and Rendering: Edition 3. New Riders. p. 442. ISBN 9780133439175. Retrieved 24 September 2017.
- Akenine-Möller, Tomas; Eric Haines; Naty Hoffman (2008). Real-Time Rendering, Third Edition: Edition 3. CRC Press. p. 11. ISBN 9781439865293. Retrieved 22 September 2017.
- Boreskov, Alexey; Evgeniy Shikin (2013). Computer Graphics: From Pixels to Programmable Graphics Hardware. CRC Press. p. 5. ISBN 9781482215571. Retrieved 22 September 2017.
- Möller, Tomas; Haines, Eric (1999). Real-Time Rendering (1st ed.). Natick, MA: A K Peters, Ltd.
- Salvator, Dave (21 June 2001). "3D Pipeline". Extremetech.com. Extreme Tech. Archived from the original on 17 May 2008. Retrieved 2 Feb 2007.
- Malhotra, Priya (July 2002). Issues involved in Real-Time Rendering of Virtual Environments (Master's). Blacksburg, VA: College of Architecture and Urban Studies. pp. 20–31. Retrieved 31 January 2007.
- Haines, Eric (1 February 2007). "Real-Time Rendering Resources". Retrieved 12 Feb 2007.
- RTR Portal – a trimmed-down "best of" set of links to resources