Computer Graphics for Virtual  
and Augmented Reality  
Lecture 06 Introduction to  
Augmented Reality  
Edirlei Soares de Lima  
<edirlei.lima@universidadeeuropeia.pt>  
What is Augmented Reality?  
From Reality to Virtual Reality:  
Mixed Reality  
Real  
World  
Augmented  
Reality (AR)  
Virtual  
Reality (VR)  
Virtual  
World  
Augmented reality combines real and virtual  
images.  
Interactive in real-time  
Virtual objects are fixed in the space of the real  
world.  
Augmented Reality  
Main characteristics:  
Combination of real and virtual elements  
Real-time updates at interactive speed  
Interaction and integration between objects  
Goal of AR:  
Make users believe that the virtual elements are part of the real world.  
Interaction and navigation:  
Virtual Reality: users interact and navigate using real world metaphors  
Augmented Reality: users interact and navigate in the real world  
Augmented Reality  
AR uses a feedback loop between human user and computer  
system.  
The user observes the AR display and controls the viewpoint.  
The system tracks the user’s viewpoint, registers the pose in the real  
world with the virtual content, and presents situated visualization.  
Registration  
Situated  
of virtual  
content  
visualization  
Virtual content  
SPATIAL MODEL  
User input  
and camera  
movement  
Pose  
tracking  
Real world model  
Applications of Augmented Reality  
Visualize building projects  
Interior design and decoration  
Applications of Augmented Reality  
Manufacturing and maintenance  
Logistics operations  
Applications of Augmented Reality  
Discrepancy Analysis  
Infrastructure Inspection  
Applications of Augmented Reality  
Advertising  
Live language translation  
Applications of Augmented Reality  
Sports and news  
Tourism  
Applications of Augmented Reality  
Medicine  
Fashion and dressing  
Applications of Augmented Reality  
Parking Assistant  
Navigation  
Applications of Augmented Reality  
Games  
Instructions and guides  
Strong vs. Weak AR  
Weak AR  
Imprecise tracking  
No knowledge of the environment  
Limited interactivity  
Handheld AR  
Strong AR  
Very accurate tracking  
Seamless integration into the real  
world  
Natural interaction  
Head mounted AR  
Computer Graphics for AR  
AR requires less computer graphics power  
Only the virtual objects need to be rendered in real-time  
AR requires better calibration and registration  
Tracking must be precise to guarantee that objects stay the same  
places in the real world while the users moves.  
AR requires a good composition of real and virtual elements  
A seamless integration of the virtual objects into the real world  
requires color and light adjustments.  
Technologies for VR Systems  
Display  
Optical see-through  
Additive blending  
Video see-through  
Computer calculated blending  
Projection-based  
Multiplicative blending  
Tracking  
Localization  
Finding the location  
Alignment  
Registering virtual objects to the real world  
Interaction  
Touching and manipulating virtual objects  
Display Technologies for VR  
See-Through  
Displays  
Monocular  
Binocular  
Video  
Optical  
Video  
Optical  
See-Through  
See-Through  
See-Through  
See-Through  
Single  
Camera  
Dual  
Camera  
Monoscopic  
Overlays  
Stereoscopic  
Overlays  
Example  
Products  
Monoscopic  
Overlays  
Stereoscopic  
Overlays  
Monoscopic  
Overlays  
Stereoscopic  
Overlays  
E.g.: smartphone- or E.g.: Microvision  
tablet-based  
hand-held AR  
Also: Google Glass in  
VST mode  
Nomad,  
DigiLens DL40,  
TacEye ST,  
E.g.: Vuzix iWear VR920  
with Possible, but no E.g.: Canon COASTAR,  
E.g.: Microsoft HoloLens,  
Epson Moverio BT-200,  
E.g.: Trivisio  
ARVision  
Vuzix M2000AR  
iWear CamAR  
clear advantage Vuzix Wrap 1200DXAR E.g.: Lumos DK-40 Vuzix STAR 1200XLD  
Optical See-Through Display  
Optical see-through head mounted display  
The real world is visible  
Only virtual objects are rendered and displayed  
Uses an optical element to combine a user’s view  
of the real world with computer-generated  
images  
Generated Virtual Imagery  
Pose Sensors  
(
potentially stereoscopic)  
Real World  
Optical  
Combiners  
Optical See-Through Display  
Optical See-Through Display  
Characteristics:  
Additive optical combination  
Accommodation issues  
Focus on both real and virtual objects  
Typically, poor field-of-view  
Examples:  
Optical See-Through Display  
Composition in optical see-through displays:  
Combination of light from display and real world  
Does not occlude and cannot show black  
+
=
Optical See-Through Display  
Composition in optical see-through displays:  
There are some solutions for occlusion  
Kiyokawa et al. An Occlusion-Capable Opt
Co-located Collaboration, 2003.  
Optical See-Through Display  
Accommodation problem:  
The lens projects the display at a  
certain distance  
The user’s eyes must focus on both  
real and virtual objects at a distance.  
Multi distance displays:  
Varying lenses  
Multiple waveguides + different lenses  
Magic leap: 6 guides = 2 depths  
Challenges  
Transparency  
Transition between depths  
Video See-Through Display  
Real world is shown as video  
Captures the real world with a video camera  
Overlay (combine) video feed with virtual objects  
Pose Sensors (optional)  
Image  
Sensors  
Digital  
Combiner  
Monitors  
Generated  
Virtual Imagery  
(
potentially  
stereoscopic)  
Video See-Through Display  
Video See-Through Display  
Characteristics:  
Good video quality is required  
Resolution/color/contrast in  
camera and display  
Latency/refresh-rates in camera  
and display  
Matchable time delays  
Cyber sickness problems  
Field-of-view  
Frustum and perspective  
must match between camera  
and computer graphics  
Image Quality Comparison  
Image quality in optical see-through displays is higher for the  
real world, but generally inconsistent.  
Video See-Through Display  
Video Monitor AR:  
Non-co-located display  
Augmented mirror or TV shows  
Projection-Based AR  
Spatial AR  
User does not wear equipment.  
Project virtuality onto real objects  
May apply compensation for uneven geometry, color and lighting  
Generated  
Virtual Imagery  
Projector  
Image  
Sensors  
Pose Sensors  
(optional)  
Projection-Based AR  
Projection-Based AR  
Re-map projection from a point-of-view  
Can potentially improve view for a group of viewers (any  
number of simultaneous viewers)  
Full resolution of the real world  
Nothing to wear  
Sensitive to lighting and material conditions  
Occlusion and geometry issues  
Display Systems Classification  
Technologies for VR Systems  
Display  
Optical see-through  
Additive blending  
Video see-through  
Computer calculated blending  
Projection-based  
Multiplicative blending  
Tracking  
Localization  
Finding the location  
Alignment  
Registering virtual objects to the real world  
Interaction  
Touching and manipulating virtual objects  
Localization  
Localization:  
Finding the location and surroundings  
Required for looking up models, annotation...  
Geolocation: GPS, WiFi, etc.  
Local fine-tuning by other tracking methods.  
Alignment  
Alignment:  
Registering virtual objects to the  
real world (position and  
orientation)  
Requires high fidelity tracking:  
Use knowledge from the real  
world  
Estimate the camera posture  
relative the real world  
Integration and Interaction  
Integration:  
Virtual objects optically interacting  
Correct lighting conditions  
Occlude virtual objects according real-world objects  
Cast shadows onto real objects (and vice versa)  
Interaction:  
Touching and manipulating virtual objects  
Virtual objects colliding with real objects  
Real objects affected by virtual  
Optical Interaction  
Virtual objects occluding real objects  
Easy with video see-through  
Real objects occluding virtual objects  
Mask using model of real world  
Requires a model of the real world  
Physical Interaction  
Virtual objects colliding with real world  
objects  
Real world knowledge + alignment  
Requires a model of the real world  
Real objects affected by virtual objects  
Motors, pneumatics, etc...  
Further Reading  
Schmalstieg, D., Hollerer, T. (2016). Augmented Reality: Principles and  
Practice (1st ed.). Addison-Wesley Professional.  
Chapter 1: Introduction to Augmented Reality  
Chapter 2: Displays