Narrative Dramatization

Narratives and Interactive

Storytelling

Lecture 03 – Narrative Dramatization

Edirlei Soares de Lima

<edirlei.lima@universidadeeuropeia.pt>

Narrative Dramatization

•

What is narrative dramatization?

–

The visual representation of a narrative.

–

There are many ways to create a visual representation for a narrative

using different media formats (text, images, videos, 2D/3D animations,

comics, virtual reality, augmented reality…)

Dramatization Methods: Text

•

Text:

–

Involves the translation the logical events

of the narrative into natural language

sentences.

Example: kidnap(Draco, Marian)

•

“... the princess Marian was kidnap by the

terrible villain Draco...”

It requires natural language processing

techniques to guarantee the logical

coherence of the generated narrative text.

Examples of systems:

•

Tale-Spin (1977)

Universe (1984)

Minstrel (1992)

•

Simplified solution: text templates

Templates for Text Generation

•

Save(CH1, CH2)

–

“After a great act of bravery, CH1 saves the life of CH2.”

Save(Brian, Marian)

–

“After a great act of bravery, Brian saves the life of Marian.”

Save(Hoel, Marian)

–

“After a great act of bravery, Hoel saves the life of Marian.”

Dramatization Methods:

2D/3D Computer Graphics

2D/3D environments:

•

–

Involves the translation the logical events

of the narrative into actions performed by

virtual characters.

–

Actions are represented by animations and

character movements.

Challenges:

Examples of systems:

•

How to control characters (movement and

behaviors)?

How to control cameras dynamically?

How to create engaging visual

dramatizations?

•

Façade (2002)

Madame Bovary (2003)

Logtell (2010)

•

Heavy Rain (2010)

Camera Control in Interactive Storytelling

•

The cameras should work like the cameras used in games?

No!

–

It requires more robust camera control methods.

–

The camera must behave like the ones used in films.

D

A

B

C

E

Lima, E.S., et al. Virtual

Cinematography Director for

Interactive Storytelling. International

Conference on Advances in Computer

Entertainment Technology (ACE 2009).

Camera Control

Feature 1 Feature 2 Feature 3 Feature 4 ...

Best camera

Cam B

Sample 1 A1_happy A2_happy A1_talking A1_xPos

Sample 2 A1_happy A2_sad A2_talking A1_xPos

.. ...

...

Cam C

.

•

The system can be trained by

human experts (directors,

editors...)

Results:

–

Accuracy: 98%

Processing time: < 1 ms

Video

Visual/Audio Effects

Fear

Sadness

Anger

•

The system can be trained by human experts

directors of photography...)

(

Results:

–

Acurracy: 96%

Processing time: < 1 ms

Video

Lima, E.S., et al. Director of Photography and

Tension

Music Director for Interactive Storytelling. IX

Brazilian Symposium on Computer Games and

Digital Entertainment (SBGames 2010).

Interactive Comics

•

2D dramatization using the

style of comic books;

Lima, E.S., et al. Non-Branching Interactive Comics.

International Conference on Advances in Computer

Entertainment Technology (ACE 2013).

Interactive Comics

(

1) Events:

A, B, C, D, E, F, H, I, J

•

2D dramatization using the

style of comic books;

:

(2) Panels:

[P₁] [P₂]

[P₄][P₅][P₆]

[

P₃]

:

(3) Weights:

3

2

5

:

2 2 2

:

Page definition:

[

R₂]

^[^R3^]

(4) Rows:

[R₁]

–

Event grouping;

:

(5) Weights:

5

:

6

–

Panel size;

4

80

(6) Page:

[F₁]

:

16

P1^w=3

A,B

P2^w=2

C

(7) Weight:

R₁^w=5

R₂^w=5

P₃^w=5

D,E,F

P₄^w

P₅^w

=2

I

P₆^w

=2

J

R₃^w=6

=2

H

Lima, E.S., et al. Non-Branching Interactive Comics.

International Conference on Advances in Computer

Entertainment Technology (ACE 2013).

F₁^w=16

Interactive Comics

•

2D dramatization using the

style of comic books;

Page definition:

–

Event grouping;

–

Panel size;

•

Panel compositing:

–

Character placement;

–

Speech balloon placement;

Lima, E.S., et al. Non-Branching Interactive Comics.

International Conference on Advances in Computer

Entertainment Technology (ACE 2013).

Dramatization Methods:

Augmented Reality

•

Involves the translation of the logical

events of the narrative into actions

performed by virtual characters in a

augmented reality environment.

•

Actions are represented by animations

and character movements.

Challenges:

–

How to control characters (movement and

behaviors)?

How the virtual objects/characters interact

with the real world?

Examples of systems:

AR Façade (2006)

• wiz Qubes (2008)

Paper and Pencil IS (2014)

•

User interaction?

Dramatization Methods:

Virtual Reality

•

Involves the translation of the logical

events of the narrative into actions

performed by virtual characters in a

virtual reality environment.

•

Actions are represented by animations

and character movements.

Challenges:

–

How to control characters (movement and

behaviors)?

How users interact with the virtual

objects/characters?

Examples of systems:

•

Madame Bovary on the

Holodeck (2007)

–

•

Deep Space (2009)

Dramatization Methods: Videos

•

Involves the translation of the logical

events of the narrative into actions

performed by real actors.

Actions are represented by video

segments.

–

Pre-recorded videos;

–

Automatically composed videos;

•

Challenges:

Examples of systems:

Accidental Lovers (2006)

• Last Call (2010)

Deliver me to Hell (2010)

• The Princess Kidnapping (2011)

•

–

How to add interactivity to static video

segments?

•

–

How to reduce the amount of videos to

shot?

•

Modern Little Red Riding

Hood (2014)

Video-Based Interactive Storytelling

•

Real-time video compositing:

Lima, E.S. Video-Based Interactive Storytelling. PhD Thesis. Pontifícia Universidade

Católica do Rio de Janeiro (PUC-Rio), Rio de Janeiro, Brazil, August, 2014.

Video-Based Interactive Storytelling

•

Camera Control:

–

Artificial Neural Networks:

Video-Based Interactive Storytelling

Project Task:

Dramatization with Comics

StorylineA – Without user interaction

…

^Pi

P_i₊₁

Storyline B – User interacts and changes the girl’s decision

^Pi

P_i₊₁

Windows: http://www.inf.puc-rio.br/~elima/is/ComicsViewer.zip

MacOS: http://www.inf.puc-rio.br/~elima/is/ComicsViewer.app.zip

Comics Viewer: Context Overview

•

General structure:

All events of the story.

.

/Events>

Edges>

..

Connections between

the story events.

<

...

<

/StoryNetwork>

Interactions>

/Edges>

Definition of the user

interaction points.

<

.

/Interactions>

Resources>

..

Images to represent

characters, objects and

environments.

<

...

<

/Resources>

Operators>

Definition of how each

event is visually

represented.

...

<

/Operators>

/ComicsViewer>

<

Comics Viewer: Events

•

Events>

Events:

<

.

/Events>

Event id = "ID" event = "EVENT1(PARAM1, PARAM2, ...), ..."/>

..

•

Examples:

<Event id = "N2" event = "give(Grandmother, red covering,

Little girl, Grandmother's house)"/>

<

Event id = "N3" event = "ask_to_take(Mother, Little girl,

basket of food, Grandmother, Mother's house)"/>

Event id = "N4" event = "tell-right(Mother, Little girl,

<

[But remember, don't&talk to strangers and take&care of

yourself.&], Mother's house)"/>

<

Event id = "N5" event = "go(Little girl, Mother's house,

the crossroad), go(Little girl, Mother's house, the woods),

go(Little girl, Mother's house, villain's house)"/>

Comics Viewer: Edges

•

Edges>

Edges:

<

.

/Edges>

Edge startevent = "ID1" endevent = "ID2"/>

..

•

Examples:

Comics Viewer: Resources

•

Resources>

Resources:

<

.

/Resources>

Resource name = "NAME" file = "FILENAME"/>

..

•

Examples:

<

Resource name = "Grandmother-Give" file = "images/Grandmother.png"/>

Resource name = "red covering" file = "images/red covering.png"/>

Resource name = "Little girl-Recv" file = "images/girl_recv.png"/>

Resource name = "Mother" file = "images/Mother.png"/>

Resource name = "Mother's house" file = "images/Mother's_bg.png"/>

Resource name = "the crossroad" file = "images/crossroad_bg.png"/>

Comics Viewer: Resources

•

Image resources:

–

Images of characters and objects must have transparent backgrounds

i.e. no background);

Separated images are required to represent different actions of the

same character (with different visual poses);

(

The default resolution of the comics panels is 400 x 210;

Examples: http://www.inf.puc-rio.br/~elima/is/IS_IMG_Pack.zip

Comics Viewer: Operators

•

Operators:

<

.

Element type = "T" resource = "R" x = "X" y = "Y" scale = "S"/>

..

</Operator>

</Operators>

•

Example 1:

<Element type = "image" resource = "#B#" x = "0" y = "0"

scale = "1"/>

<Element type = "image" resource = "#A#-Go" x = "100" y = "20"

scale = "0.5"/>

</Operator>

Comics Viewer: Operators

•

Example 2:

<

Element type = "image" resource = "#D#" x = "0" y = "0"

scale = "1"/>

Element type = "image" resource = "#A#-Give" x = "30" y = "20"

scale = "0.5"/>

Element type = "image" resource = "#B#" x = "30" y = "20"

scale = "0.5"/>

Element type = "image" resource = "#C#-Receive" x = "170" y = "20"

scale = "0.5"/>

Element type = "balloon" resource = "#C#, this&#B# is&for you!&"

x = "200" y = "30" mx = "140" my = "75" mw = "20"/>

</Operator>

Project Assignment 3

3

) Create a comics dramatization for the interactive narrative of

your project (the same narrative created in the last Project

Assignment) using the Comics Viewer system.

–

User interactions are not required for this assignment (this is a task for

the next project assignment).