Scalable, Controllable, Efficient and Convincing Crowd Simulation

1. Scalable, Controllable, Efficient and Convincing Crowd Simulation Mankyu Sung University of Wisconsin-Madison

2. What is Crowd? • Crowd • A group of individuals • Share a same environment • Have a same goal but may act in different way Unique Natural Phenomenon

3. Crowd Simulation Artificial creation of virtual crowds Collection of behaviors of each individual Behaviors Selection of motions over time What is Crowd Simulation? Real Crowd Virtual Crowd

4. Challenges • Highly complex crowd behaviors • Ex) Street environment : crossing, sitting, window-shopping, walking with friends… • Not easy to maintain real-time performance • Real time animation of more than 1,000 characters is hard • Not easy to achieve convincingness • Realistic motions, Realistic behaviors • Need to minimize the control parameters

5. Research Goal : • Four demands for crowd simulation Scalability Controllability Crowd simulation method that meets demands Our goal Convincingness Efficiency

6. (1) Scalability • Definition:Ability of crowd simulation to continue to function well when the environment become more complicated. • Scalable Character Memory • Agent’s memory is not proportional to the complexity of environment. • Scalable Authoring • The whole simulation should be broken into a set of small simulations. • Author new simulation through simple copy&paste

7. (2) Controllability • Definition:Ability to specify particular behaviors to characters by setting constraints • Constraints • High-level scenario for controlling crowd flow • No specific individual • Low-level individual constraints • Specific individual

8. (3)Efficiency • Definition:Fast processing of algorithm • Fast motion synthesis • Fast collision detection • Linear simulation performance

9. (4)Convincingness • Definition:Ability to cause viewers to believe specific aspects of crowds. • Visually convincing behavior • No discontinuity on motion • No collisions between characters • Semantically convincing behavior • No unreasonable behaviors (depending on animator’s intention)

10. Outline • Features of crowds • Two-level crowd simulations method • Probability scheme • Collision detection through MOBB(Motion Oriented Bounding Box) – (Briefly) • Constrained motion synthesis • Summary and future work

11. Solution : Observe real crowdsWhat features crowds have? • Anonymity • Hard to distinguish one from others • It does not matter who is doing, but what is happening there matters • Locality • Behaviors depend on location and social relationship • Variety • People behave differently

12. Solution :Two-Level crowd simulation framework Situations (What should each individual do ?) Situations (What should each individual do ?) High-level Motion Synthesis (How to show motions?) Low-level

13. An Environment Example: Model of Street Simulation: Real time, Reactive Rendering: Unreal Game Engine for playback

14. In a hurry: Check for traffic Run across street Observation:Behavior Depends on Situation Store Window Doorway In front of Store Window Possibly: stop to window shop In front of Doorway Possibly: open door, enter Unlikely: stand blocking door Sidewalk: Walk here Friends Together: Possibly: Stop to talk Probably: Have same goal Use crosswalk: Wait for green light Start crossing In Crosswalk: Walk across street once you’ve started Street: Generally, don’t walk here

15. Store Window Doorway In front of Store Window Possibly: stop to window shop In front of Doorway Possibly: open door, enter Unlikely: stand blocking door Sidewalk: Walk here Friends Together: Possibly: Stop to talk Probably: Have same goal Use crosswalk: Wait for green light Start crossing Managing Environmental Complexity:Situation-Based Approach • Many different situations • An agent only needs a few at a time • Each situation has a different set of local behaviors • Plugs information into agents for local behaviors • Scalable authoring and scalable behaviors

16. Collision Avoidance (Sung et al CS-TR-05-1592) Probability Scheme (Sung et al EG 04) Constrained Motion Synthesis (Sung et al SCA 05) Solution :Two-Level crowd simulation framework Situations (Sung et el, EG 04) High-level Motion Synthesis Low-level

17. Low-level : Two motion synthesis techniques • Probability Scheme (Sung et al EG 04) • Stochastic crowd control • Short term view of agents • Constrained Motion Synthesis(Sung et al SCA 05) • Planning-based behaviors • Have constraints for individuals Use Snap-Together-Motion (Gleicher et al I3D 03)

18. Related Work:Inspirations from Research • Crowd Modelling(e.g. Reynolds 1987, Musse 2001, Blue 1998, Henderson, 1974, Bouvier et al. 1997, Helbing et al 2000, …) • Simple agents, simple rules • Fast, scalable, emergent patternsbut…difficult to generalize to complex behaviors • Smart Environments(e.g. Kallman et al 1998, Farenc et al. 1999, Thomas et al 2000, Michael et al 2003, …) • Objects tell agents what to do • Scalable authoring, complex behaviorsbut… difficult to compose or get variability

19. Related WorkInspirations from Systems • The Sims(Maxis/EA Games) • Smart environment/ Smart object • Scalable authoring • Does not consider the social relations • Commercial softwares • Massive, SoftImage/Behavior, Character Studio, AI-Implant… • Author scenes, not environments • Similar means, Different ends • Agent-centric approach

20. Solution : Probability Scheme Situations(Sung et el, EG 04) High-level Collision Avoidance (Sung et al CS-TR-05-1529) Probability Scheme (Sung et al EG 04) Probability Scheme (Sung et al EG 04) Constrained Motion Synthesis (Sung et al SCA 05) Low-level

21. Probability scheme : Agent Architecture • Agents: • Discrete set of actions (from mograph) • Randomly choose from distributions • Behavior functions provide distributions • All aspects of agents can be updated dynamically

22. Probability Scheme:Simple Default Agents • Default agents very simple • Wander, don’t bump into things, … • Extend agents as necessary to achieve complex behaviors

23. Situations:Extensible Agent • Situations extend agents • Add Actions • Add Behavior Functions • Add Sensors and Rules that inform Behavior Functions

24. Example • Default agent can’t cross the street. • How an agent crosses the street… • Enters a Crosswalk Situation • Crosswalk situation extends agent • Sensor to see traffic lights • Behavior Functions to cross a street • Behavior Functions to stop • Rules to wait for the light to change • Remove extensions when done

25. Composing Behaviors:Action Selection Left ? Agent Right Straight

26. Composing Behaviors:Probability Scheme Behavior Function A Left .5 Agent Right .3 Straight .2

27. Composing Behaviors:Probability Scheme Behavior Function A Behavior Function B .4 Left .5 .1 Agent .25 Right .3 .1 .33 Straight .2 .2

28. Situations Compose • Agent can be in multiple situations • Agent has union of all the things that different situations put in

29. Authoring: Painting Interface • Author environments (not characters) • Set of situation types • Paint into environments • Mix situations to make complex/compound ones

30. Experiments • Gallery environment • Street environment

31. Solution :Collision Avoidance Situations(Sung et al, EG 04) High-level Collision Avoidance (Sung et al CS-TR-05-1529) Probability Scheme (Sung et al EG 04) Constrained Motion Synthesis (Sung et al SCA 05) Low-level

32. Collision Detection :MOBB(Motion Oriented Bounding Box) • Problem : Collision test between motions • A motion is a sequence of poses (frames) • Solution : 3D Bounding box of motion trajectory (X, Y, time) t t y y x x

33. root level1 level1 level2 level2 level2 level2 Level 2 Level 1 Level 3 MOBB Trees :Hierarchical subdivision of motions

34. MOBB Trees :Construction of MOBB trees • Local Axis-Aligned Method v1 frame_end v2 Motion trajectory d Bounding Sylinder v3 frame_start v4

35. A00 B00 A10 B10 A11 B11 MOBB Trees : Intersection tests • Given two trees, intersections are tested from the top node to the bottom node until they reach leaf nodes MOBB(A) MOBB(B) A11 B11 A10 B10

36. Experiments

37. Solution :Constrained Motion Synthesis Situations(Sung et al, EG 04) High-level Collision Avoidance (Sung et al CS-TR-05-1529) Probability Scheme (Sung et al EG 04) Constrained Motion Synthesis (Sung et al SCA 05) Constrained Motion Synthesis (Sung et al SCA 05) Low-level

38. The Goal:Motion Synthesis for Crowds Orientation • Problem : Constrained motion synthesis • Positions, Orientation, Poses, Time duration • Requirement • Fast performance • Accurate meeting constraints • High quality motions • Complicated environment Pose Position Target Time duration Initial

39. Our approach : • Synthesize crowds one individual at a time • Structured Motion graphs for low-level synthesis (Gleicher et al. ’03) Must adapt to crowds • Individual motions must be found very quickly • Pure discrete synthesis cannot meet continuous constraints

40. Adapting Graph based Synthesis : • Two-level synthesis • Coarse search for global path planning • Finer search for detailed motion synthesis • Quickly find long motions in complex environments • Incorporate continuous motion adjustment • Discrete search to roughly satisfy constraints • Additional displacements for precision • Improves speed and accuracy

41. Related Work (1) • Graph based motion synthesis(e.g. Arikan2002, Arikan 2003, Gleicher 2003, Kovar 2002, Hue 2004, Lee 2002, Lee 2004, Reitsma 2004) • Connecting discrete finite clips with simple interpolation or displacement mapping -Create new motions strictly by attaching clips →Hard to satisfy constraints exactly - Do not consider crowds

42. Related Work (2) • Procedural Motion Synthesis (Bouvier 1997, Boulic 1990, Sun 2001, Boulic 2004) • Controllable but not as realistic as motion capture data • Motion Blending (Guo 1996, Park 2004, Petteré 2003) • Continuous control over trajectory • Limited and computationally costly

43. Rough planning PRM query Fine planning Greedy search Create seed motions If distance > ε Randomly select and replace a clip Joining with adjustment Example Target Obstacle waypoints Algorithm Initial

44. Rough planning PRM query Fine planning Greedy search Create seed motions If distance > ε Randomly select and replace a clip Joining with adjustment Example Algorithm Target Obstacle Initial 1 2 3 waypoints

45. Rough planning PRM query Fine planning Greedy search Create seed motions If distance > ε Randomly select and replace a clip Joining with adjustment Example Algorithm Target Obstacle Forward Motion(Mf) Initial 1 Backward Motion(Mb) 2 3 Initial’

46. Rough planning PRM query Fine planning Greedy search Create seed motions If distance > ε Randomly select and replace a clip Joining with adjustment Algorithm Cost function : How close are they? C(Mf, Mb) > ε Forward motions Backward motions Compare all pair of motions and return minimum cost

47. Rough planning PRM query Fine planning Greedy search Create seed motions If distance > ε Randomly select and replace a clip Joining with adjustment Algorithm < ε New motions Old Motions Old Motionsc Random select and Replace a clip

48. Rough planning PRM query Fine planning Greedy search Create seed paths If distance > ε Randomly select and replace a clip Joining with adjustment Example Algorithm Target Obstacle Initial Joining waypoints

49. Demos Spelling out SCA Time constrained demo Box delivery Big crowds

50. Performance results