Interaction Methods for Virtual Reality in Unity

Computer Graphics for Virtual

and Augmented Reality

Lecture 04 – Interaction Methods for

Virtual Reality in Unity

Edirlei Soares de Lima

<edirlei.lima@universidadeeuropeia.pt>

Interaction Methods for Virtual Reality

•

Interaction with a virtual world is a key ingredient of a VR

experience.

•

Selection and Manipulation:

–

Selection methods allow users to select one or more objects from a

set

–

Manipulation methods allow users to modifying object properties

•

Position, orientation, scale, shape, color, texture, behavior, etc.

Navigation:

–

Defines how users move from place to place within the virtual

environment

Selection Techniques

•

Simple Virtual Hand:

–

Direct mapping between physical and virtual hands

Object can be selected by “touching” with the virtual hand

Limitation: user can only select objects that can be reached by hand

https://www.youtube.com/watch?v=a4RxqoG093A

Virtual Hand in Unity

•

Adding virtual hands models:

–

Create an XR Rig in the scene: GameObject -> XR ->

XR Origin;

–

3D hand model:

•

https://developer.oculus.com/downloads/package/oculus-

hand-models/ (Maya files)

•

http://edirlei.com/aulas/vrar/3DHandsFBX.zip (FBX files)

–

Add the virtual hands as child objects of Left- and

Right-Hand Controllers in the XR Rig.

•

Adjust the rotation and position of hands to match the

position orientation of the real hands (must be tested in VR).

Virtual Hand in Unity

•

Grab Animation:

–

Create “Idle”, “Grab” and “Activate” animations

using Unity’s Animation System.

•

Use the same hand model to reuse the same

animations on both left and right hands.

–

Create an animator and state machine to

control the animation transitions.

•

Use boolean parameters for the transitions.

Virtual Hand in Unity

public class HandAnimation : MonoBehaviour{

public InputActionProperty grabAction;

public string grabAnimationParameter;

public InputActionProperty activateAction;

public string activateAnimationParameter;

public Animator handAnimator;

private bool grabbing;

private bool activating;

private void Start(){

grabbing = false;

activating = false;

}

private void Update(){

if (handAnimator)

{

float grabButton = grabAction.action.ReadValue<float>();

float activateButton = activateAction.action.ReadValue<float>();

Virtual Hand in Unity

if (grabButton > 0)

{

if (!grabbing)

{

grabbing = true;

handAnimator.SetBool(grabAnimationParameter, grabbing);

}

else if (grabbing)

{

grabbing = false;

handAnimator.SetBool(grabAnimationParameter, grabbing);

}

if (activateButton > 0)

{

if (!activating)

{

activating = true;

handAnimator.SetBool(activateAnimationParameter, activating);

}

Virtual Hand in Unity

else if (activating){

activating = false;

handAnimator.SetBool(activateAnimationParameter, activating);

}

Direct Hand Interaction in Unity

•

XR Interaction Toolkit implementation:

–

Remove the default XR Ray Interactor

components from the hand controllers.

–

Add a XR Direct Interactor and Collider

component to the hand controllers.

•

The collider must be set to trigger.

–

Create the interactive objects to be activated

by hand interaction (buttons and grabbable

cube).

•

Add a collider and rigid body to the objects.

Direct Hand Interaction in Unity

•

The grabbable cube must have XR

Grab Interactable Component.

The buttons will use a custom script

based on the XRBaseInteractable.

Direct Hand Interaction in Unity

public class InteractiveButton : XRBaseInteractable

{

private bool buttonActivated;

private void Start(){

activated.AddListener(ActivateButton);

buttonActivated = false;

GetComponent<Renderer>().material.color = Color.white;

}

private void ActivateButton(ActivateEventArgs arg0){

buttonActivated = !buttonActivated;

if (buttonActivated){

GetComponent<Renderer>().material.color = Color.blue;

}

else{

GetComponent<Renderer>().material.color = Color.white;

}

Selection Techniques

•

Ray-casting:

–

Simulates a “laser pointer” attached to the virtual hand

First object intersected by the ray can be selected

Perform well for remote selection

https://www.youtube.com/watch?v=OVrd-KMD-dM

Ray Casting Interaction in Unity

•

XR Interaction Toolkit implementation:

–

Add a XR Ray Interactor component to the hand

controllers.

–

Add a XR Interactor Visual Line component to

the hand controllers.

•

Optional component to draw the ray cast line.

–

Create the interactive objects to be activated by

hand interaction (buttons and grabbable cube).

•

Add a collider and rigid body to the objects.

Use the XR Grab Interactable Component in the Cube.

The buttons use the same script created before.

Selection Techniques

•

Occlusion:

–

Image-plane technique (2D)

Occlude/cover object with the selector object (e.g. hand or finger)

Nearest object along a ray from the eye through selector object can be

selected

Selection Techniques

•

Go-Go (Arm-extension):

–

Non-linear mapping between physical and virtual hand position

–

Can access local and distant objects

Selection Techniques

•

Other common selection techniques:

–

Cone-casting

Snapping

3D Bubble Cursor

Sphere-casting

PRISM

ARM

A survey of 3D object selection techniques for VR:

–

https://doi.org/10.1016/j.cag.2012.12.003

Assignment 1

1

) Implement the interaction process to open and close a door

in Virtual Reality.

–

When the door is closed and the user grabs the door handle, the door

must open.

When the door is open and the user grabs the door handle, the door

must close.

Free 3D door models:

https://assetstore.unity.com/packages/3d/p

rops/interior/door-free-pack-aferar-148411

Manipulation Techniques

•

HOMER:

–

Selection: ray-casting

Manipulate: directly with virtual hand

Include linear mapping to allow wider range of placement in depth.

https://www.youtube.com/watch?v=V6Fo3iza5cY

Manipulation Techniques

•

World-in-miniature:

–

A miniature of the world held in the user’s hand

Miniature objects can be manipulated directly

Moving miniature objects affects full-scale objects

Can also be used for navigation

https://www.youtube.com/watch?v=dTzIyNHJ4jE

https://www.youtube.com/watch?v=zOTx_CWsR7g

Manipulation Techniques

•

Scaled-World Grab:

–

At selection, scale user up (or world down) so that virtual hand is

actually touching selected object

Assignment 2

2) Implement the world-in-miniature manipulation technique.

–

The miniature of the world can be presented in front of the user.

–

The use must be able to select and manipulate the miniature objects

directly.

–

Moving miniature objects must affect full-scale objects.

•

Hint: create a script to connect the miniature objects with the full-scale objects.

Changes to the position and orientation of the miniature object must be reflected

in the connected full-scale object.

https://www.youtube.com/watch?v=dTzIyNHJ4jE