Computer Graphics for Virtual  
and Augmented Reality  
Lecture 01 Introduction to  
Computer Graphics  
Edirlei Soares de Lima  
What is Computer Graphics?  
The term computer graphics describes  
any use of computers to create and  
manipulate images [Marschner, S., et  
al., 2015].  
Computer graphics is the science and  
art of communicating visually via a  
computer’s display and its interaction  
devices [Hughes, J. F., et al., 2013].  
What is Computer Graphics?  
Computer graphics is a cross-disciplinary field:  
Physics (e.g.: model light behavior);  
Mathematics (e.g.: describe shapes);  
Human Perception (e.g.: only render things that will be  
Human-Computer Interaction (e.g.: interaction devices);  
Engineering (e.g.: optimize allocation of memory, and  
processor time);  
Graphic Design and Art (e.g.: make the computer-to-  
human communication more effective);  
What is Computer Graphics?  
Geometric Modeling  
Rendering and  
Computer Vision  
Image Processing  
Computer Graphics Areas  
Modeling: Deals with the mathematical specification of shape  
and appearance.  
Definition of surface, interpolation rules, reflection models.  
Computer Graphics Areas  
Rendering: Term inherited from art create shaded images  
from 3D computer models.  
Many techniques and styles of rendering.  
Computer Graphics Areas  
Animation: technique to create an illusion of motion through  
sequences of images.  
Time is a key issue, not usually dealt with in basic modeling and  
Computer Graphics Areas  
Image processing: deals with the manipulation of 2D images  
and is used in both the fields of graphics and vision.  
Computer Graphics Areas  
Virtual reality: attempts to immerse the user into a 3D virtual  
Typically requires at least stereo graphics and response to head  
Computer Graphics Areas  
Augmented reality: brings 3D virtual elements to the real  
Computer Graphics Areas  
3D scanning: uses range-finding technology to create  
measured 3D models.  
Computer Graphics Areas  
Computational photography: explores the use of computer  
graphics, computer vision, and image processing methods to  
enable new ways of photographically capturing objects,  
scenes, and environments.  
Applications Video Games  
Video games increasingly use sophisticated 3D models and  
rendering algorithms.  
Applications Cartoons  
Cartoons are often rendered directly from 3D models.  
Many traditional 2D cartoons use backgrounds rendered from 3D  
Applications Visual Effects  
Visual effects use almost all types of computer graphics  
Superimpose backgrounds with separately filmed foregrounds.  
Synthetic environments, objects, and even characters.  
Applications Animated Films  
Animated films use many of the same techniques that are  
used for visual effects, but without necessarily aiming for  
images that look real.  
Applications CAD/CAM  
Computer-Aided Design and Computer-Aided Manufacturing  
use computer technology to design parts and products on the  
computer to guide the manufacturing process.  
Applications Simulation  
Simulation can be thought of as accurate video gaming.  
Applications Medical Imaging  
Medical imaging creates meaningful images of scanned  
patient data.  
Applications Information Visualization  
Information visualization creates images of data that do not  
necessarily have a “natural” visual representation.  
Graphics Hardware  
Computer hardware and software (graphics card and device  
driver) that generates computer graphics and allows them to  
be shown on a display.  
The processors on GPUs are highly parallel and support  
thousands of concurrent threads.  
The hardware is designed to process larger numbers of pixels and  
vertices in short amounts of time.  
Graphics APIs  
A graphics API (Application Program Interface) is a set of  
functions that perform basic operations such as drawing  
images and 3D surfaces into windows on the screen.  
Graphics APIs vs User-Interface APIs.  
Dominant Graphics APIs: Direct3D and OpenGL.  
Graphics Pipeline  
The Graphics Pipeline is a special software/hardware  
subsystem that efficiently draws 3D primitives on screen.  
Is optimized for processing 3D triangles with shared vertices.  
The basic operations in the pipeline map the 3D vertex  
locations to 2D screen positions and shade the triangles so  
that they both look realistic and appear in proper back-to-  
front order.  
Geometric manipulation using matrices and vectors.  
The speed at which images can be generated depends  
strongly on the number of triangles being drawn.  
One of the basic tasks of computer graphics is rendering  
three-dimensional objects.  
Taking a scene, or model, composed of many geometric objects  
arranged in 3D space and producing a 2D image that shows the  
objects as viewed from a particular viewpoint.  
D Objects  
2D Image  
Rendering involves considering how each object contributes  
to each pixel.  
Example of 3D data (.obj file format):  
v -0.402936 -0.094247 0.152067  
v -0.145051 -0.129821 0.323038 vertices  
v 0.156651 -0.106614 0.313985  
vt 0.321778 0.604842  
vt 0.305640 0.547919  
vt 0.317529 0.539270  
texture mapping  
vn -0.7273 0.6863 0.0002  
vn -0.8335 0.5525 -0.0009  
vn -0.7615 0.4492 0.4671  
vertex normals  
f 217/1/1 397/2/2 178/3/3 58/4/4  
f 11/5/5 169/6/6 184/7/7 62/8/8  
f 165/9/9 10/10/10 59/11/11 180/12/12  
faces (vertex index/texture index/  
normal index)  
Rendering can be performed in two general ways:  
Image-order rendering: each pixel is considered in turn, and for each  
pixel all the objects that influence it are found and the pixel value is  
for each pixel do  
for each object do  
Object-order rendering: each object is considered in turn, and for each  
object all the pixels that it influences are found and updated.  
for each object do  
for each pixel do  
Graphics Pipeline  
Graphics Pipeline  
Vertex Processing and Primitive Processing:  
Input: vertex and attributes;  
Vertex Assembly;  
Modeling, Viewing, and Projection Transformations;  
Backface Culling;  
Graphics Pipeline  
Fragment generation;  
Multiple possible fragments per pixel;  
Interpolation attributes along each primitive;  
Graphics Pipeline  
Fragment Processing:  
Compute color of each fragment;  
Compute depth of each fragment;  
Per-fragment Shading;  
Texture Mapping;  
Remove hidden surfaces (z-buffer algorithm);  
Graphics Pipeline  
Pixel Processing:  
The various fragments corresponding to each pixel  
are combined;  
Write the output image in the frame buffer;  
Graphics Pipeline Example  
vertices = {v0x, v0y, v0z,  
v1x, v1y, v1x,  
v2x, v2y, v2z,  
v3x, v3y, v3x,  
v4x, v4y, v4z,  
Texture map  
v5x, v5y, v5x}  
texture_coords = {v0u, v0v,  
v1u, v1v,  
v2u, v2v,  
v3u, v3v,  
v4u, v4v,  
v5u, v5v}  
Graphics Pipeline Example  
Step 1: Transform triangle vertices into camera space.  
Graphics Pipeline Example  
Step 2: Apply perspective projection to transform triangle  
vertices into a normalized coordinate space (canonical view  
Graphics Pipeline Example  
Step 3: Discard triangles that lie complete outside the  
canonical view volume and clip triangles that extend beyond  
the canonical view volume (possibly generating new triangles).  
Graphics Pipeline Example  
Step 4: Transform vertex positions from the canonical view  
volume into screen coordinates (x, y).  
Graphics Pipeline Example  
Step 5: break each primitive into a number of fragments, one  
for each pixel covered by the primitive.  
Graphics Pipeline Example  
Step 6: perform a texture mapping or other more advanced  
shading computation per fragment.  
Graphics Pipeline Example  
Step 7: compute the depth of each fragment (z-buffer  
Graphics Pipeline Example  
Step 8: update the frame buffer.  
Graphics Pipeline  
The graphics pipeline is simply a way to describe the  
functioning of a standard graphics system.  
The exact implementation of the graphics pipeline or even the order  
in which the tasks are performed may vary.  
The pipeline can be seen as a black box, but some  
understanding of the nature of the processing is valuable.  
Currently, the fixed-function pipeline model is being  
replaced by shaders.  
However, the fixed-function pipeline makes a good conceptual  
framework where we can add variations, which is how most shaders  
are in fact created.  
OpenGL vs Direct3D  
Unity Rendering Pipeline  
Unity supports two main rendering paths:  
Forward Rendering: renders each object in one or more passes,  
depending on lights that affect the object.  
Is based on the traditional linear graphics pipeline, where each geometry is  
processed by the pipeline (one at a time) to produce the final image.  
Deferred Rendering: renders each object once on the first pass and  
stores shading information into G-buffer textures. Additional passes  
compute lighting based on G-buffer and depth in screen space.  
The rendering is "deferred" until all of the geometries have been processed by the  
pipeline. The final image is produced by applying shading/lightning at the end.  
Forward Rendering  
Deferred Rendering  
Forward vs. Deferred  
Deferred rendering is better for lighting:  
In a standard forward rendering pipeline, the lighting calculations have  
to be performed on every vertex and on every fragment in the visible  
scene, for every light in the scene.  
Forward: O(number_of_fragments * number_of_lights)  
Deferred: O(number_of_pixels * number_of_lights)  
Deferred rendering problems:  
No support for anti-aliasing;  
No support for semi-transparent objects;  
Not supported by old video cards and mobile devices.  
Further Reading  
Hughes, J. F., et al. (2013). Computer Graphics: Principles and Practice  
3rd ed.). Upper Saddle River, NJ: Addison-Wesley Professional. ISBN: 978-  
Chapter 1: Introduction  
Marschner, S., et al. (2015). Fundamentals of Computer Graphics (4th  
ed.). A K Peters/CRC Press. ISBN: 978-1482229394.  
Chapter 1: Introduction  
Chapter 3: Raster Images