Introduction to Computer Graphics

Computer Graphics for Virtual

and Augmented Reality

Lecture 01 – Introduction to

Computer Graphics

Edirlei Soares de Lima

<edirlei.lima@universidadeeuropeia.pt>

What is Computer Graphics?

•

The term computer graphics describes

any use of computers to create and

manipulate images [Marschner, S., et

al., 2015].

Computer graphics is the science and

art of communicating visually via a

computer’s display and its interaction

devices [Hughes, J. F., et al., 2013].

What is Computer Graphics?

•

Computer graphics is a cross-disciplinary field:

–

Physics (e.g.: model light behavior);

Mathematics (e.g.: describe shapes);

Human Perception (e.g.: only render things that will be

noticed);

–

Human-Computer Interaction (e.g.: interaction devices);

Engineering (e.g.: optimize allocation of memory, and

processor time);

–

Graphic Design and Art (e.g.: make the computer-to-

human communication more effective);

What is Computer Graphics?

Geometric Modeling

Data

Rendering and

Computer Vision

Animation

Image

Image Processing

Computer Graphics Areas

•

Modeling: Deals with the mathematical specification of shape

and appearance.

–

Definition of surface, interpolation rules, reflection models.

Computer Graphics Areas

•

Rendering: Term inherited from art – create shaded images

from 3D computer models.

–

Many techniques and styles of rendering.

Computer Graphics Areas

•

Animation: technique to create an illusion of motion through

sequences of images.

–

Time is a key issue, not usually dealt with in basic modeling and

rendering.

Computer Graphics Areas

•

Image processing: deals with the manipulation of 2D images

and is used in both the fields of graphics and vision.

Computer Graphics Areas

•

Virtual reality: attempts to immerse the user into a 3D virtual

world.

–

Typically requires at least stereo graphics and response to head

motion.

Computer Graphics Areas

•

Augmented reality: brings 3D virtual elements to the real

world.

Computer Graphics Areas

•

3D scanning: uses range-finding technology to create

measured 3D models.

Computer Graphics Areas

•

Computational photography: explores the use of computer

graphics, computer vision, and image processing methods to

enable new ways of photographically capturing objects,

scenes, and environments.

Applications – Video Games

•

Video games increasingly use sophisticated 3D models and

rendering algorithms.

Applications – Cartoons

•

Cartoons are often rendered directly from 3D models.

–

Many traditional 2D cartoons use backgrounds rendered from 3D

models.

Applications – Visual Effects

•

Visual effects use almost all types of computer graphics

technology.

–

Superimpose backgrounds with separately filmed foregrounds.

–

Synthetic environments, objects, and even characters.

Applications – Animated Films

•

Animated films use many of the same techniques that are

used for visual effects, but without necessarily aiming for

images that look real.

Applications – CAD/CAM

•

Computer-Aided Design and Computer-Aided Manufacturing

use computer technology to design parts and products on the

computer to guide the manufacturing process.

Applications – Simulation

•

Simulation can be thought of as accurate video gaming.

Applications – Medical Imaging

•

Medical imaging creates meaningful images of scanned

patient data.

Applications – Information Visualization

•

Information visualization creates images of data that do not

necessarily have a “natural” visual representation.

Graphics Hardware

•

Computer hardware and software (graphics card and device

driver) that generates computer graphics and allows them to

be shown on a display.

The processors on GPUs are highly parallel and support

thousands of concurrent threads.

–

The hardware is designed to process larger numbers of pixels and

vertices in short amounts of time.

Graphics APIs

•

A graphics API (Application Program Interface) is a set of

functions that perform basic operations such as drawing

images and 3D surfaces into windows on the screen.

–

Graphics APIs vs User-Interface APIs.

Dominant Graphics APIs: Direct3D and OpenGL.

Graphics Pipeline

•

The Graphics Pipeline is a special software/hardware

subsystem that efficiently draws 3D primitives on screen.

–

Is optimized for processing 3D triangles with shared vertices.

The basic operations in the pipeline map the 3D vertex

locations to 2D screen positions and shade the triangles so

that they both look realistic and appear in proper back-to-

front order.

–

Geometric manipulation using matrices and vectors.

•

The speed at which images can be generated depends

strongly on the number of triangles being drawn.

Rendering

•

One of the basic tasks of computer graphics is rendering

three-dimensional objects.

–

Taking a scene, or model, composed of many geometric objects

arranged in 3D space and producing a 2D image that shows the

objects as viewed from a particular viewpoint.

Rendering

Process

3

D Objects

2D Image

•

Rendering involves considering how each object contributes

to each pixel.

Rendering

•

Example of 3D data (.obj file format):

v -0.402936 -0.094247 0.152067

v -0.145051 -0.129821 0.323038 ← vertices

v 0.156651 -0.106614 0.313985

...

vt 0.321778 0.604842

vt 0.305640 0.547919

vt 0.317529 0.539270

←

texture mapping

coordinates

...

vn -0.7273 0.6863 0.0002

vn -0.8335 0.5525 -0.0009

vn -0.7615 0.4492 0.4671

←

vertex normals

...

f 217/1/1 397/2/2 178/3/3 58/4/4

f 11/5/5 169/6/6 184/7/7 62/8/8

f 165/9/9 10/10/10 59/11/11 180/12/12

←

faces (vertex index/texture index/

normal index)

Rendering

•

Rendering can be performed in two general ways:

–

Image-order rendering: each pixel is considered in turn, and for each

pixel all the objects that influence it are found and the pixel value is

computed.

for each pixel do

for each object do

...

–

Object-order rendering: each object is considered in turn, and for each

object all the pixels that it influences are found and updated.

for each object do

for each pixel do

...

Graphics Pipeline

•

Vertex Processing and Primitive Processing:

–

Input: vertex and attributes;

Vertex Assembly;

Modeling, Viewing, and Projection Transformations;

Clipping;

Backface Culling;

Graphics Pipeline

•

Rasterization:

–

Fragment generation;

Multiple possible fragments per pixel;

Interpolation attributes along each primitive;

Graphics Pipeline

•

Fragment Processing:

–

Compute color of each fragment;

Compute depth of each fragment;

Per-fragment Shading;

Texture Mapping;

Remove hidden surfaces (z-buffer algorithm);

Graphics Pipeline

•

Pixel Processing:

–

The various fragments corresponding to each pixel

are combined;

–

Write the output image in the frame buffer;

Graphics Pipeline – Example

•

Input:

–

vertices = {v0x, v0y, v0z,

v1x, v1y, v1x,

v2x, v2y, v2z,

v3x, v3y, v3x,

v4x, v4y, v4z,

Texture map

v5x, v5y, v5x}

texture_coords = {v0u, v0v,

v1u, v1v,

v2u, v2v,

v3u, v3v,

v4u, v4v,

v5u, v5v}

Graphics Pipeline – Example

•

Step 1: Transform triangle vertices into camera space.

Graphics Pipeline – Example

•

Step 2: Apply perspective projection to transform triangle

vertices into a normalized coordinate space (canonical view

volume).

Graphics Pipeline – Example

•

Step 3: Discard triangles that lie complete outside the

canonical view volume and clip triangles that extend beyond

the canonical view volume (possibly generating new triangles).

Graphics Pipeline – Example

•

Step 4: Transform vertex positions from the canonical view

volume into screen coordinates (x, y).

Graphics Pipeline – Example

•

Step 5: break each primitive into a number of fragments, one

for each pixel covered by the primitive.

Graphics Pipeline – Example

•

Step 6: perform a texture mapping or other more advanced

shading computation per fragment.

Graphics Pipeline – Example

•

Step 7: compute the depth of each fragment (z-buffer

algorithm).

Graphics Pipeline – Example

•

Step 8: update the frame buffer.

Graphics Pipeline

•

The graphics pipeline is simply a way to describe the

functioning of a standard graphics system.

–

The exact implementation of the graphics pipeline or even the order

in which the tasks are performed may vary.

•

The pipeline can be seen as a black box, but some

understanding of the nature of the processing is valuable.

Currently, the fixed-function pipeline model is being

replaced by shaders.

–

However, the fixed-function pipeline makes a good conceptual

framework where we can add variations, which is how most shaders

are in fact created.

OpenGL vs Direct3D

OpenGL

Direct3D

Unity Rendering Pipeline

•

Unity supports two main rendering paths:

–

Forward Rendering: renders each object in one or more passes,

depending on lights that affect the object.

•

Is based on the traditional linear graphics pipeline, where each geometry is

processed by the pipeline (one at a time) to produce the final image.

–

Deferred Rendering: renders each object once on the first pass and

stores shading information into G-buffer textures. Additional passes

compute lighting based on G-buffer and depth in screen space.

•

The rendering is "deferred" until all of the geometries have been processed by the

pipeline. The final image is produced by applying shading/lightning at the end.

Forward Rendering

Deferred Rendering

Forward vs. Deferred

•

Deferred rendering is better for lighting:

–

In a standard forward rendering pipeline, the lighting calculations have

to be performed on every vertex and on every fragment in the visible

scene, for every light in the scene.

•

Complexity:

–

Forward: O(number_of_fragments * number_of_lights)

–

Deferred: O(number_of_pixels * number_of_lights)

Deferred rendering problems:

–

No support for anti-aliasing;

No support for semi-transparent objects;

Not supported by old video cards and mobile devices.