How to synthesize temporal visual signals as transient input traffic for

1. How to synthesize temporal visual signals as transient input traffic for discrete event simulation? Synthesizing Transient Traffic of Temporal Visual Signals for Discrete Event Simulation Mauritz Panggabean and Leif Arne Rønningen Department of Telematics, NTNU 3rd International Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) 2011 Budapest, Hungary, 5-7 October 2011

2. Outline • Introduction • Background and vision • Framework: the big picture • Why transient traffic? • Preparation and experiment • Type of input: the scenes • Processing the clips • Modeling using linear and power functions • Results: synthetic transient traffic using power function • Conclusion and the next work

3. Begin with the end in mind. Stephen Covey Background and vision • Futuristic tele-immersive environment system for real-time artistic collaborations with near-natural quality • Arrays of auto-stereoscopic 3D displays and high-end cameras on all surfaces of the collaboration space • More than one performers in a collaboration space • Multiview with view activated by the gaze of the performers • Challenge:modeling such environment in discrete event simulation prior to construction • Simulation for feasibility study with (un)compressed input data • First step: a model for synthetic transient input traffic from human motion and camera modes

4. If everyone is thinking alike, then somebody isn't thinking. George S. Patton Framework: the big picture Stochastic and deterministic human motion Humanoid model Synthesis of multiobject human motion Synthetic multiobject human body motions Formal model Computation of projected areas based on eye gazes Synthetic eye gazes Synthesis of eye gazes Uncompressed visual traffic: Multiview projected body-areas based on eye gazes Incorporating effects from image compression Rate-distortion curves of image compression Discrete-event simulation of complex multiview multiobject distributed tele-immersive collaboration system Compressed visual traffic: Multiview projected body-areas based on eye gazes

5. Have you got a problem? Do what you can where you are with what you've got. Theodore Roosevelt Why transient traffic? • Stationary traffic nullified by interframe compression • Important: transient slopes, variations, extreme values and duration (not in mean values) • Piecewise analysis: a general formal model of a few parameters to synthesize each piece as input traffic (GOAL)

6. Everything should be made as simple as possible, but not simpler. Albert Einstein Types of input: the scenes • Only one static camera, one view and one object • Three scenes as input video clips: • Clip PANNING • Object enters the scene from left and disappears on the right side • Equivalent to camera panning • Clip ZOOM • Object gradually moves closer to the camera • Equivalent to camera zoom • Clip MOTION • The object performs some motions with the limbs at the center of the scene

7. It is an acknowledged truth in philosophy that a just theory will always be confirmed by experiment. Thomas Malthus Processing the clips Original color clips (1920x1080, 30Hz) Transcoding De-interlacing - Off-the-shelf camera - Interlaced, uncompressed - One person - Uniform color background AVI color clips (1280x720, 30Hz) Uncompressed input traffic Object segmentation - Object only (the person) - 8 bits per pixel - RGB channels - Matlab - PC (2.99GHz, 8.0 GB RAM)

8. No amount of experimentation can ever prove me right; a single experiment can prove me wrong. Albert Einstein Input traffic from the clips 20 25 30 35 40 45 1 50 100 150 200 250 1 25 55 118 127 145

9. In much of society, research means to investigate something you do not know or understand. Neil Armstrong Modelling as power function Power functionf(x) = axband linear functionf(x) = cx + d with normalized frame number a = 1.108, b = 0.8518 RMSE = 0.09655 c = 1.116, d = 0.03048 RMSE = 0.104 a = 0.9533, b = 1.339 RMSE = 0.03234 c = 0.9854, d = -0.08577 RMSE = 0.04202 a = 1.134, b = 0.8762 RMSE = 0.1677 c = 1.145, d = 0.02072 RMSE = 0.172 Power function f(x) = axb as the general model for synthesis

10. Research is to see what everybody else has seen, and to think what nobody else has thought. Albert Szent-Gyorgyi Power function for transient input traffic synthesis in simulation • Transient traffic with increasing trend f(x) = axb + d + e • Transient traffic with decreasing trend f(x) = a(1-xb) + d + e • Parameters: • b, to control the curve bending: 0.5 < b < 1 • a = Dmax– Dmin, d = Dmin: Dmin and Dmaxas the min/max data • e, to control the random smoothness of the curve at point x. emin < e < emaxis uniformly distributed, emin ≤ emax and e < a. • Frame rate F and simulation time S (sec): t = 1/F and x = t/S

11. An experiment disproving a prediction is a discovery. -Enrico Fermi Synthetic transient input traffic For all emin= 0 emax = 0.5 S = 1 Panning a = 4 b = 0.65 c = 0 F = 19 Zoom a = 12 b = 1.34 c = 8 F = 230 Motion Increasing part a = 3, b = 0.75, c = 7 F = 9 Decreasing part a = 2.5, b = 2.5, c = 7.5 F = 11 Adjacent pieces of arbitrary transient traffic connected by setting their values of a and c

12. Research is what I'm doing when I don't know what I'm doing. -Wernher von Braun Conclusion and the next work Stochastic and deterministic human motion Humanoid model Synthesis of multiobject human motion Synthetic multiobject human body motions • Power function as a general formal model for transient traffic synthesis as input for discrete event simulation • Covers human motion, camera panning and zoom for one view Formal model Computation of projected areas based on eye gazes Synthetic eye gazes Synthesis of eye gazes Uncompressed visual traffic: Multiview projected body-areas based on eye gazes Incorporating effects from image compression Rate-distortion curves of image compression Discrete-event simulation of complex multiview multiobject distributed tele-immersive collaboration system Compressed visual traffic: Multiview projected body-areas based on eye gazes

13. The real voyage of discovery consists not in seeking new landscapes, but in having new eyes. - Marcel Proust Thank you