180 likes | 187 Views
Synergies between World Opera and Internet of Things. Frank Eliassen - Lucas Luiz Provensi - Roman Vitenberg Lille, 10.03.2010. Topics. World Opera Gilgamesh Production Scenarios Location tracking Motion recognition Light Control Multimedia sensor networks Information services
E N D
Synergies between World Opera and Internet of Things Frank Eliassen - Lucas Luiz Provensi - Roman Vitenberg Lille, 10.03.2010
Topics • World Opera • Gilgamesh Production • Scenarios • Location tracking • Motion recognition • Light Control • Multimedia sensor networks • Information services • Challenges
World Opera • Worldwide distributed operas • Singers that are remotely located can interact as if they are in the same stage. • Musicians distributed around the world can play together. • Audience can see the opera from one of the stages or from other remote place.
Gilgamesh • First World Opera Production (2012) • 3 Stages: Tromsø – Stockholm – Copenhagen; • Based on The Epic Of Gilgamesh, an epic poem from old Mesopotamia.
Gilgamesh • Virtual Stages Uruk Gilgamesh Cedar Forest Land of the Dead Humbaba Enkidu
Gilgamesh - Observations • The number of multimedia flows/streams coming in and going out will be different for each stage • It can change at runtime as well • At certain times, distributed actors may be interacting with each other while the musicians at all stages keep playing • Actor has a fixed assignment to a physical stage but may travel between virtual stages • A physical avatar to represent the actor at the others stages, like a puppet with a display in the face that shows the video stream of the face of the actor
Observations: Video vs Audio • Musicians are very sensitive to audio quality but do not care too much about video quality • Remote musicians are sensitive to maestro acting. They are not very tolerant to delay in the maestro's video streaming • When acting against distributed peers, actors are not tolerant to delay and they need a good video quality and enough resolution to see face details • Actors also need to hear audio, e.g., when in a synchronized dialog • Embed a speaker into the avatar so that the actors have the sensation that the voice is coming from that point
Location tracking • The maestro will be at the city stage, and he will represent the sun god and will be walking around the stage • The conductor location will be tracked to physically reproduce the movement of the maestro at the other stages.
Location tracking • Position of actors in a virtual world.
Movement Recognition Wii Mote (Nintendo) Composed by accelerometer and optical sensor. Can be extended with a gyroscope.
Movement Recognition Project Natal (Microsoft) Depth sensor consisting of an infrared projector combined with a monochrome CMOS sensor
Movement recognition • Replicate musicians movements on virtual entities or phenomena.
Movement recognition • Replicate actors movements on virtual avatars
Light control • Lighting sensors can provide information lighting condition to scene controller software.
Scenarios • Multimedia sensor networks • New sources of media content : views of different angles and audio collected at different points.
Services • Services offered to the audience at the stages and outside. • Information about the ongoing act; • Access to alternative media streams; • Shared comments with other users. Act 2 Gilgamesh and Enkidu fights the Bull of Heaven Bull Gilgamesh Enkidu
Challenges Delay Synchronity Adaptation Dependability Scalability
Thank you! • Questions?