Real-time computer graphics
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages)(Learn how and when to remove this template message)
Real-time computer graphics or real-time rendering is the subfield of computer graphics focused on producing and analyzing images in real-time. The term is most often used in reference to interactive 3D computer graphics, typically using a graphics processing unit (GPU). Video games are its most notable application. The term can also refer to anything from rendering an application's GUI (graphical user interface) to real-time image processing and image analysis.
Computers have been known since their invention to be capable of generating 2D images involving simple lines, images, and polygons in real-time (for instance, using Bresenham's line drawing algorithm). Quickly and effectively rendering 3D objects on computer systems, however, has always been a daunting task for traditional Von Neumann architecture-based systems. Before true 3D graphics could be displayed, sprites were used to give the illusion of 3D graphics from 2D images.
Different techniques for rendering now exist, such as ray-tracing and rasterization. Computation and rendering can now be generated at a speed that allows for a series of rendered images to induce the illusion of movement. This illusion allows meaningful user interaction with the software. So, the calculations for image rendering can take into account user input.
Principles of real-time 3D computer graphicsEdit
The goal of computer graphics is to generate a computer generated image using certain desired metrics. This image is often called a frame. How fast these images or frames are generated in a given second, determines the method's real timeliness. Real-time computer graphics is different from traditional offline rendering systems (and hence, these are the non-real-time graphics systems), non-real-time graphics typically rely on ray-tracing where the expensive operation of tracing rays from the camera to the world is allowed and can take as much as hours or even days for a single frame.
On the other hand, in the case of a real-time graphics, the system has less than 1/30th of a second per image. In order to do that, the current systems cannot afford shooting millions or even billions of rays; instead, they rely on the technique of z-buffer triangle rasterization. In this technique, every object is decomposed into individual primitives, the most popular and common one is the triangle. These triangles are then 'drawn' or rendered onto the screen one by one. Each triangle gets positioned, rotated and scaled on the screen and a special hardware (or in the case of an emulator, the software rasterizer) called rasterizer generates the pixels inside each of these triangles. These triangles are then decomposed into further smaller atomic units called pixels (or in computer graphics terminology, aptly called fragments) that are suitable for displaying on a display screen. The pixels are then drawn on the screen using a certain color; current systems are capable of deciding the color that results in these triangles—for e.g. a texture can be used to 'paint' a triangle, which is simply deciding what color to output at each pixel based on a stored picture; or in a more complex case, at each pixel, one can compute if a certain light is being seen or not resulting in very good shadows (using a technique called shadow mapping).
Video game graphicsEdit
Real-time graphics is oriented toward providing as much quality as possible for the lowest performance cost possible for a given class of hardware. Most video game and simulators fall in the category of real-time graphics. As mentioned above, real-time graphics is currently possible due to the significant recent advancements in these special hardware component called graphics processing units (GPUs). These GPUs are capable of handling millions of triangles per frame. Current DirectX 11/OpenGL 4.x class hardware is capable of generating complex effects on the fly (i.e. in real-time) such as shadow volumes, motion blurring, real-time triangle generation are among many others. Although the gap in quality between real-time graphics and traditional off-line graphics is narrowing, the accuracy is still far below the accuracy of offline rendering. The advancement of real-time graphics is evidenced in the progressive improvements between actual gameplay graphics and the pre-rendered cutscenes that are typically found in modern video games. As hardware has advanced, cutscenes are now typically rendered in real-time. Advances in 3D game engines in the mid 1990s saw users experience greater interactivity.
Another difference between real-time and non-real-time graphics is the interactivity desired in real-time graphics. Feedback is typically the main motivation for pushing real-time graphics to its furor. In cases like films, the director has the complete control and determinism of what has to be drawn on each frame, typically involving weeks or even years of decision-making involving a number of people.
In case of a real-time interactive computer graphics, usually a user is in control of what is about to be drawn on the display screen; the user typically uses an input device to provide feedback to the system—for example, wanting to move a character on the screen—and the system decides the next frame based on this particular instance of action. Usually the display is far slower (in terms of the number of frames per second) in responsiveness than the input device (in terms of the input device's response time measured in ms). In a way this is justified due to the immense difference between the infinitesimal response time generated by the human-being's motion and the very slow perspective speed of the human-visual system, this results in significant advancements in computer graphics, whereas the advancements in input devices typically take much longer time to achieve the same state of fundamental advancement (e.g., the current Wii Remote), as these input devices have to be extremely fast in order to be usable.
Another important factor controlling real-time computer graphics is the combination of physics and animation. These techniques largely dictate what is to be drawn on the screen—or more precisely, where to draw certain objects (deciding their position) on the screen. These techniques imitate the behavior (the temporal dimension, not the spatial dimensions) seen in real-world to a degree that is far more realistic than and compensating computer-graphics' degree of realism.
Real-time previewing within graphics software, especially when adjusting lighting effects, allows increase in work speed. Some parameters adjustments in fractal generating software may be adjusted while viewing changes to the image in real-time.
Graphics rendering pipelineEdit
Graphics rendering pipeline is known as the rendering pipeline or simply the pipeline, and it is the foundation of real-time graphics. Its main function is to generate, or render a two-dimensional image, given a virtual camera, three-dimensional objects (an object that has width, length, and depth), light sources, lighting models, textures and more.
The architecture of the real-time rendering pipeline can be divided into three conceptual stages: Application, Geometry, and Rasterization. This structure is the core which is used in real-time computer graphics applications.
The application stage is driven by the application where it begins the image generation process that results in the final scene of frame of animation, therefore creating a base filled with simple images, that can later on, build up into a bigger, more clear image. The application is implemented in the software thus giving the developers total control over the implementation in order to change and/or improve the performance. This stage may, for example, contain collision detection, speed-up techniques, animations, force feedback, etc. One of the processes that is usually implemented in this stage is collision detection. Collision detection usually includes algorithms that detects two objects collision. After a collision is detected between two objects, a response may be generated and sent back to the colliding objects as well as to force a feedback device. Other processes implemented in this stage includes, Texture Animation, Animations via transforms, Geometry Morphing, or any kind of calculations that are not performed in any other stages. At the end of the application stage, which is also the most important part of this stage, the geometry to be rendered is fed to the next stage in the rendering pipeline. These are the rendering primitives that might eventually end up on the output device, such as points, lines, and triangles, etc.
The geometry stage is responsible for the majority of the per-polygon operations or per-vertex operation, which means that this stage computes what is to be drawn, how it should be drawn, and where it should be drawn. A polygon mesh is created. This stage is usually done of specialised hardware or graphics processing units. In some cases, this stage might be defined as one pipeline stage or several different stages, mainly due to the different implementation of this stage. However, in this case, this stage is further divided into different functional groups.
Model and view transformEdit
Before the final model is shown on the output device, the model is transformed into several different spaces or coordinate systems. That is, when an object is being moved or manipulated, the object's vertices are what are being transformed.
In order to give the model a more realistic appearance, one or more light sources are usually equipped during the scene of transforming the model. However, this stage cannot be reached without completing the 3D scene being transformed into the view space; the view space is where the camera is placed at the origin and aimed in a way that the camera is looking in the direction of the negative z-axis, with the y-axis pointing upwards and the x-axis pointing to the right.
There are two types of projection, orthographic (also called parallel) and perspective projection. Orthographic projection is used to represent a 3D model in a two dimensional (2D) space. The main characteristic of orthographic projection is that the parallel lines remain parallel even after the transformation without distorting them. Perspective projection is when a camera is farther away from the model, the smaller the model it appears. Essentially, perspective projection is the way that we see things from our eyes.
Clipping is the process of removing primitives that are outside of the view box in order to continue on to the rasterizer stage. Primitives that are outside of the view box are removed or "clipped" away. Once the primitives that are outside of the view box are removed, the primitives that are still inside of the view box will be drawn into new triangles to be proceeded to the next stage.
The purpose of screen mapping, as the name implies, is to find out the coordinates of the primitives that were determined to be on the inside of the view box in the clipping stage.
Once all of the necessary steps are completed from the two previous stages, all the elements, including the lines that have been drawn and the models that have been transformed, are ready to enter the rasterizer stages. Rasterizer stage means turning all of those elements into pixels, or picture elements, and adding color onto them.
- Spraul, V. Anton (2013). How Software Works: The Magic Behind Encryption, CGI, Search Engines and Other Everyday Technologies. No Starch Press. p. 86. ISBN 1593276664. Retrieved 24 September 2017.
- Wolf, Mark J. P. (2008). The Video Game Explosion: A History from PONG to Playstation and Beyond. ABC-CLIO. p. 86. ISBN 9780313338687. Retrieved 24 September 2017.
- Birn, Jeremy (2013). Digital Lighting and Rendering: Edition 3. New Riders. p. 442. ISBN 9780133439175. Retrieved 24 September 2017.
- Akenine-Möller, Tomas; Eric Haines; Naty Hoffman (2008). Real-Time Rendering, Third Edition: Edition 3. CRC Press. p. 11. ISBN 9781439865293. Retrieved 22 September 2017.
- Boreskov, Alexey; Evgeniy Shikin (2013). Computer Graphics: From Pixels to Programmable Graphics Hardware. CRC Press. p. 5. ISBN 9781482215571. Retrieved 22 September 2017.
- Möller, Tomas; Haines, Eric (1999). Real-Time Rendering (1st ed.). Natick, MA: A K Peters, Ltd.
- Salvator, Dave (21 June 2001). "3D Pipeline". Extremetech.com. Extreme Tech. Retrieved 2 Feb 2007.
- Malhotra, Priya (July 2002). Issues involved in Real-Time Rendering of Virtual Environments (Master's). Blacksburg, VA: College of Architecture and Urban Studies. pp. 20–31. Retrieved 31 January 2007.
- Haines, Eric (1 February 2007). "Real-Time Rendering Resources". Retrieved 12 Feb 2007.
- RTR Portal – a trimmed-down "best of" set of links to resources