Game Design

1. Game Design LESSON #6: Virtual Reality Day!

2. TODAY: • Playtest Workplace Games, and team Unity production time • Introduction to VR/AR • Intro to Unity for VIVE VR dev • Unity Physics Considerations • Niche Games • Final Project Planning

3. PART 2: Introduction to VR/AR

4. Introduction to VR/AR 1. VIRTUAL REALITY: A headset which completely obscures vision, providing a fully immersive visual/audio experience. • Occulus Rift (Facebook): Headset and controllers, run by computer, spatial and audio tracking, but no space delineation. • Occulus Gear VR (Samsung): Samsung phone in $100 headset. Full pivot range, controller or reticule tracking/charging, no spatial or audio tracking. • HTC VIVE (Valve): Headset, bases, and controllers. Run by computer, spatial and audio tracking with space delineation. • PlayStation VR (Sony): VR headset for Sony Playstation. • Cardboard (Google): Simple Cardboard structure to hold phones. Requires app. Gen 2 includes buttons on sides for greater interaction.

7. Introduction to VR/AR 2. AUGMENTED REALITY: Holograms or labels superimposed over real vision. Tracking lets holograms relate to real-world elements. • Vuforia (Qualcomm): The current standard for phone-based AR. Needs an image on a piece of paper or screen to create holograms that track around that image. • ARcode (Google) and ARkit (Apple): Next gen systems, need S8 and iphone 8 machines. • Hololens (Microsoft): major leap forward on Kinect body/spatial tracking tech. Offers a small window in center view, but excellent audio tracking and calculates full room mesh capture, to create holograms which integrate and interact fully with the existing space.

9. Hololens Game, Holohack 2016Hemos and Globinshttps://www.youtube.com/watch?v=i3KBfenhhJE

10. Introduction to VR/AR 3. CONTROLS: • Headset Buttons: return to menu, control pad, focus adjustment • Reticule Charging: line up reticule with HUD buttons or in game objects, hold for sufficient interval to “charge/load” to activate movement or event. • Controllers: typically one per hand, including triggers and touchpad. Haptic responses. • Hand Gestures (Hololens): pinching, swiping, etc.

11. Introduction to VR/AR 4. BODY TRACKING: • Leap Motion: Hand Tracking device on table or attached to VR visor. • Xbox Kinect:infrared sensors track motion, interprets skeleton. • Motion Capture Rigs: Full-body dot-suits in rooms with installed sensors, or magnetic strip-suits and single base (like Xsense).

12. Introduction to VR/AR 5. HISTORY: 1938, Antonin Artauddescribes the illusory nature of characters and objects in theatre as "la réalitévirtuelle" A "cyberspace" is a networked virtual reality 1991: Segaannounces Sega VR headset for arcade games and the Mega Drive console 1994: Virtual Reality Modelling Language (VRML) developed 2010: Palmer Luckey designed the 1st Oculus Rift. 2013: Valve discovered and freely shared the breakthrough of low-persistence displays which make lag-free and smear-free display of VR content possible 2014: Facebook purchased Oculus VR for $2 billion, Playstation and Google announce their devices. 2015Occulus shipped 2016 HTC VIVE shipped Major Media Examples: Books like Snowcrashby Neil Stephenson and Ready Player One by Ernest Cline summarize the ideas presented in other, more original works (like Neuromancer by William Gibson)

13. Introduction to VR/AR 5. HISTORY: Major Media Examples: The concept of Augmented Reality Holograms has ben a feature of films for decades: Star Wars games and telepresence, the Minority Report interactive display, the Tony Starks development holograms in Iron Man. The idea that glasses could show us a different reality was seen in They Live.

14. Introduction to VR/AR 5. HISTORY: Major Media Examples: And, of course, the Holodeck.

15. Introduction to VR/AR 6. THE OPPORTUNITIES FOR VR/AR: Opportunities for ENTERTAINMENT: Immersive worlds are extraordinarily powerful game and storytelling spaces! Opportunities for SOCIAL GOOD: Social understanding from immersively inhabiting the other. Opportunities for BUSINESS: • Big Ticket Sales: Buildings and construction plans with surround and site-based visualization. Furniture, Cars, other vehicle sales. • Telepresence: Long distance board meetings: see holograms of your team sitting around a table with you. • Mechanical and Medical Education: understand the structure of systems with full 3-deimensional visualization and instructions.

16. Introduction to VR/AR 7. PROBLEMS FOR VR/AR: MARKET PENETRATION: Good progress, but Studios/Content not yet sustained by sales (funded by Valve, Occulus, Microsoft, etc.) STRAIN: Too much time in VR is exhausting: holding up gear (light eyewear in the next 5 years), constant eye contact. Children under 13 not advised to use regularly (concern for developing eyes). NAUSEA: Framerate Issues:we cannot look away, so images must operate at the speed of human motion: 90 fps (90 HZ machines. NOTE: Microsoft MR may work on 60 fps/HZ). Keep polygons/ textures/ particles/ physics use low. Peripheral Vision in Motion:New content needs to be rendered as fast as it takes us to turn our heads, or we feel ill. Try horse blinders!: Black rectangles move in as we turn our head to block peripherals, usually without our noticing. Body Disassociation Problems: Any motion that does not correspond to what the mind expects the body to be doing can cause significant discomfort. Going up an elevator, or moving too fast in the virtual world (keep it to 10 feet per second).

17. Introduction to VR/AR 8. Nausea Solutions for VR: • Use controllers for teleportation and object manip, never for motion. Avoid typical controller-based motion of non-VR games. • Restrict Field of View (particularly for head-turning): put blinders (black bars) on the sides to stop peripheral vision • Keep polygon count and texture sizes low, to reduce rendering. For Vehicle Moving Games: https://uploadvr.com/five-ways-to-reduce-motion-sickness-in-vr/ 1. Reduce non-forward movements (no strafing) 2. Reduce Vection (opposite-moving train) 3. Reduce Accelerations (roller coasters are a problem) 4. Reduce camera YAW (only do head turning with actual head) 5. Add a static reference frame (like the frame around a windshield)

18. Introduction to VR/AR 8. Nausea Solutions for VR: Racing Games: Slim Rig Butt-kicker and desk fan

19. Introduction to VR/AR 9. NEW GAME MECHANICS FOR VR/AR: We are still discovering the inherent systems of these tools! Physicality: Turning the body around, like the in the Zombie AR game being created at MIT. Exploring other physical modes: • Virzoom: using an exercise bicycle as the controller (with handle-buttons) – extraordinarily motivating! • Somniacs’ Birldy: fly as a bird through a city. No discomfort, because body and eyes sending similar signals to the brain) Storytelling: Giving the audience the ultimate opportunity to decide where they look (anew kind of “three ring circus”). BoababStudios’ “Invasion!” offers an unfolding story of revelation, as the player/audience understands where they are, then who they are, then their shifting role in the narrative of a bunny and some aliens.

20. Virzoom and Birdy

21. What is VR For? The modern wave of Virtual and Augmented Reality devices and software create intensely impactful experiences that use body kinesthetics as controllers, whether through full body movement with controllers or simply by moving our heads. VR (Occulus Rift, HTC VIVE, GearVR, Google Cardboard) offers full immersion in created spaces, while AR (Hololens, Vuforia, AR Kit or Arcode running on tablets or phones) creates holograms within or superimposed on the real world. Games are an obvious use for this tech, but it also offers extraordinary storytelling possibilities, including those explored by Baobab studios, as well as uses in medicine, education, and construction, to name a few.

22. “Invasion” Immersive film by Baobab

23. What is the HTC Vive? The VIVE is immersive Virtual Reality hardware and software with Spatial Visual and Audio systems allowing us to walk around a Virtual space. The VIVE includes: • 1 Headset (tethered or free in the new versions) • 2 Controllers (and microUSB chargers): include Trigger (bottom), Touchpad (top), On-button (below touchpad on top), Menu (above touchpad on top), and Side-grip buttons. • 2 Base Stations: Need to face each other from high up (to not be blocked by players), at opposite corners of a space free of obstruction. Can be wall-mounted, but we use tripods for mobility. When stations correctly face each other they show small lit “B” & “C.” • Cords for power (Base Stations) and connecting to the computer. The computer needs a sufficient graphics card to install and run the software (at least an NVidia GEFORCE 970). The Following tutorial is based on work by Taylor Ampatiellos and Marianne Chiarella, from notes here: https://www.raywenderlich.com/9189-htc-vive-tutorial-for-unity It uses the SteamVR Interaction System to easily drag and drop key functions for making interactive experiences in Virtual Reality

24. Making 3D Models for Unity and VR • Set your Maya file to Meters. Create a scene in Maya (a living room is about 7x7 meters, for example). Use Extruded meshes (avoid unwelded Combines). Name each object. • Keep polygon counts low: try to have no objects exceed 1k tri polys. To view polygons, open Display > Heads UP Display > PolyCount, and note the triangles (“tris”) for selected objects. • Texture all objects: Create at least a solid color .PNG. Blinn materials for game objects is fine (Unity does not need Arnold materials). • File > Export Selection each object as an FBX file, naming the file for the object. NOTE: If we export each object in their original arrangements around the space, we can automatically reproduce that arrangement in Unity by dragging the objects into the Hierarchy. Kitboxing refers to the 3D game practice of making objects that are meant to be re-used in many scenes, and such objects should be exported from Maya’s world center, after Freezing Transforms.

25. [PART A] Getting the VIVE Hardware Ready To run Unity with a VIVE, first install SteamVR on the computer. • Download the Steam App:http://bit.ly/2JcrjFK. • Mouse over “Library,” find SteamVR. • RightClickand choose Install Game. This is the software that runs the VIVE hardware from the computer (syncs up the Base Stations/ Sonar/ etc), and it will only install on machines with a graphics card powerful enough to run the VIVE. NOTE:For more information see the InteractionSystem.pdf in Assets > SteamVR > InteractionSystem downloaded with the SteamVR2 API from the Unity Asset Store.

26. [PART B] Starting the SteamVR Interaction System 1 • Create a new 3D project in Unity. • Open Window > General > Asset Store or [Ctrl/Cmd]+[9]. • Search for “Steam” to find SteamVRPlugin. Download/import all components. • When prompted by a pop-up from the SteamVR system, always Accept All.

27. [PART B] Starting the SteamVR Interaction System 2 • Open the new “SteamVR” folder in your Project. Drag SteamVR > InteractionSystem > Core > Prefabs >[Player]into the Hierarchy. It has a camera, controllers, and inputs for pickups and teleportation. You can hide or replace hand meshes. • Delete MainCamera from the Hierarchy. • Add a GamObjects > 3D Objects > Cube (20, 0.1, 20) as a platform beneath [Player].

28. [PART B] Starting the SteamVR Interaction System 3 • Hit the Play button. Because it is this project’s first time playing we MUST say “yes” to some popups: • The 1st asks to Generate Actions for SteamVR Input. Select[Yes]! • 2nd says the project is missing actions.json. Select[Yes] to generate a .json. • 3rd is the SteamVR Input dialog. The Action Sets should be set to “Default.” Click[Save and Generate], close [x], and File >Save the Project. We also start seeing the window for Valve.VR.SteamVR_UnitySettyingsWindow which will appear frequently and always just needs us to click Accept All. • TESTING TIME!Turn on controllers (button below touchpad), don headset. Look/walk around and see your controllers. Be careful to stay within the hologram walls that show space limits.

29. [PART C] Picking Things Up with VIVE in Unity • Add a GameObjects > 3D Objects > Cubenear the center of the scene to be a table (set height to 0.75). Place smaller Cubes on top (0.1, 0.1, 0.1). • Add Component > Rigidbody to the small cubes for physics properties. • Drag this script onto the cubes: Assets > SteamVR > InteractionSystem > Core > Scripts >[Throwable]. This adds 3 Components to these objects, each with interesting parameters you can adjust: • “Interactable” • “Velocity Estimator” • “Throwable” • TESTING TIME! Start the scene, don headset. Move VIVE controllers near Cubes to highlight them in yellow. Squeeze triggers to pick them up. Pass cubes hand to hand, toss and catch.

30. [PART D] Teleportation To avoid nausea, player movement should be handled with teleporting or actual walking. The Player prefab already has Teleportation Inputs ready; we just need to add Teleporting to the scene: • Drag SteamVR > InteractionSystem > Teleport > Prefabs > Teleporting into the Hierarchy to set up teleport logic. • To make the floor a teleport-able area: duplicate the floor, drag the copy just above the original floor, and drag onto it Teleport > Scripts > TeleportArea. • To create specific teleport spots: drag into the scene Teleport > Prefabs > TeleportPoint and lift up until the white bottom circle is visible. • TESTING TIME! Start the scene and don your headset. Press the touchpad to see a laser emit from your controller top to target a location, and release to teleport.

31. [PART E] Target Practice • Import the Target Practice files from the course site: Ding audio and TargetDetect Script. • Add a CylinderGameObject to the scene. • Change Cylinder scale to make a thin disk. • RotateX = 90, position a short distance from the [Player] (like a dart board on a wall). • Remove the Collider Component currently on the disk. • Add Component > Physics > TWOMeshColliders to the disk. • Set one MeshCollider to be Convex and IsTrigger, to register impact for scoring/audio. • Drag the “Ding” audio file onto the disk in Hierarchy. This adds an AudioSource Component to the disk. Turn off “Play on Awake”. • Select a small cube. In the Inspector create/add a Tag called “BeanBag”. Add this Tag to all other pick-up-able cubes. • Attach script TargetDetect.cs to the disk. • TESTING TIME! Raise speaker volume, start scene, don headset. Pick up and throw cubes at the target. When you hit the target you should hear the “Ding” sound clip play!

32. [PART F] Create a Hinged Door • Create or import a Doorframe and a Door. Drag into the Hierarchy and position as desired. • To the Doorframe: Add Component > Physics > Box Collider and RigidBody. • Set Doorframe Rigidbody to IsKinematic so it does not move. • To the Door: Add Component > Physics > Box Collider, RigidBody, and also Hinge Joint. • Find “Connected Body” slot under Hinge Joint. Drag the Doorframe object from Hierarchy into that slot (will only work if Doorframe has a RigidBody and IsKinematic). • The Hinge Axis starts in a corner. Set Axis = 0,1,0 (for a side-hinge), and set X, Y, and Z positions. • Turn on Limits and set Max = 120 and Min = -120. • TESTING TIME! The door will not react to controllers alone, but if you pick up an object and hold it or throw it at the door, the door will swing on its hinge.

33. Further Exploration Teleportation and picking-up and throwing objects are just the basics. There is so much to explore! Consider these topics: • Thumb-pad interactions • Shooting enemies • Arc-based projectiles • Nausea management • Spatial and audio tracking • Getting footage into the VIVE • importing API (for Google Earth, etc) • creating collider interaction and art for the player body—head, arms, feet • Level building with all of these mechanics • Adding snap-orientation to pickup Continue your VIVE journey here: https://www.raywenderlich.com/159552/advanced-vr-mechanics-unity-htc-vive-part-1

34. PART 4: Unity Physics Considerations

35. Unity Physics: https://unity3d.com/learn/tutorials/topics/physics/physics-best-practices • Layers and Collision Matrix: to minimize physics calls, put all objects meant to be impacted by the same objects in their own layers. • Raycasts: Use as sparingly as possible, and cast as short a distance as possible. Do not use with mesh colliders– super expensive. • Physics 2D vs 3D: 3D costs more CPU– use 2D for 2D and 2.5D games. • Rigidbody: Apply to anything that needs physics, to avoid calculating physics on static colliders. • Fixed Timestep Optimization: This value in the Time Manager adjusts how FixedUpdate() manages the Physics update rate. Change this value to find a compromise between accuracy and CPU use on Physics.

36. PART 5: Game Design Method #6: Niche Games

37. QUESTION: Why Make a Game for a specific community?

38. QUESTION: Why Make a Game for a specific community? • Built-in audience. • Parameters of cause and audience inform design. • Transform Society: inform or change views about a way of life or a population, creating opportunities for healing and empathy. • Games for health: Hope/ happiness/ positivity leads to healing.

39. Niche/Health Games: Immune Attack

40. Niche/Health Games: Immune Attack • AUDIENCE: Children with immunology disorders. • SYMPTOMS: Hopelessness, isolation. • SOLUTIONS: Give the player agency: feel like they can reach inside and heal their bodies/ reactivate their immune system. Educate on relevant systems. • “You must navigate ananobot through a 3D environment of blood vessels and connective tissue in an attempt to save an ailing patient by retraining her non-functional immune cells.  Along the way, you will learn about the biological processes that enable macrophages and neutrophils – white blood cells – to detect and fight infections.”

41. Niche/Health Games: Elude

42. Niche/Health Games: Elude

43. Niche/Health Games: Elude • AUDIENCE: Family of people with bipolar disorder. • SYMPTOMS: Cannot comprehend lows or highs, of being out of control of emotional states. • SOLUTIONS: Experience the cycle: frequent crushing difficulty and rare moments of elation, to get a glimpse of the experience. • “Life is a never-ending struggle, full of rising and falling moods. Elude mirrors this struggle against the rising tide of depression, and the search for a path to happiness. Yet happiness remains elusive. Again and again, losing passion for anything in life, you plunge into depression. Only by continually calling out to the world can you find experiences that resonate, allowing you to ascend into happiness.”

44. Niche Games Alternatives Please note there are many more possible choices for niche game audiences beyond games for health: • Games to educate specific audiences, like literacy games for young children, science games for medical students, etc. How do you make an onerous learning task, one that typically pains the learner, into one that is engaging? • Games to offer parents alternatives when dealing with difficult developmental stages. For this assignment, please avoid job-specific niches, and consider wider populations.

45. Niche/Health Games: In-Class Exercise: • AUDIENCE: Brainstorm multiple causes and specific audiences. Consider games for health: child, adult, or elderly patients suffering from specific physical or mental illnesses. Consider military suffering from PTSD, women with Body Dysmorphia, or young adults with Social Anxiety Disorder. • SYMPTOMS: Do research: what are the emotional symptoms of the suffering? • SOLUTIONS: Like in our ARGs, consider specific solutions for each symptom. What do those with a specific disorder or illness, or their families, need to feel better?: To feel in control, to relinquish control, to better understand/empathize, to gain strength to fight, to relax? • What playable mechanics could offer these solutions, and lend themselves to interesting choices?

46. PART 6: Final Games:Start Concepting!

47. Final Games NEXT WEEK: We will help you decide your final games and teams: a multi-stage brainstorming exercise to help you imagine, discuss, and start revising concepts for games. You will then have the opportunity to pitch a game to the class, and everyone will vote with their feet for the project they want to create. TEAMS will be 4-6 people, between 6-8 teams. Those whose idea is accepted for development have an additional challenge, to let go of their idea almost immediately, since it will be dramatically changed every week through the iterative weekly cycle of development, testing, and revision.

48. Final Games WHAT ARE YOU EXCITED ABOUT?: Start thinking about the game you might want to make in the final two months of this course. I encourage you to consider VR and AR, but you are welcome to explore any genre or mode of interaction. INCORPORATE WHAT YOU HAVE LEARNED: Consider audiences you want to serve, the routines you want to explore, the miseries you want to address, the established systems you want to disrupt, and the types of movement, asset collection, and interactivity you want to proscribe. SCOPE: Single player or local multiplayer are fine but networked multiplayer is not recommended (huge amount of work). Similarly, a story theme is encouraged but a massive RPG or a group of different mini-games are not good choices for this course: consider a singular and unique mode of interaction that can be explored in increasingly challenging levels. These will just be starting places — we will explore more next week!

49. Final GamesWhy should people care about your game? • Make a Game You Want to Play: if you won’t enjoy it, chances are your audience won’t, ether. • Address a Real-world Problem: Solutions make news, and fascinate audiences. • Original Gameplay Mechanics (Disruption): New ways to move, to solve problems, to feel and think. • Original Visuals and Audio: Create new visual experiences with radically playful and wacky art. Your pitches will include: the premise and gameplay, with a focus on the core player verbs. What can a player DO in this game? What are the unnecessary obstacles to their success?

50. Final Game Ideas We Suggest Avoiding Narrative Heavy Games, like RPGs:Narrative Design for Games is a big topic that we are not covering in this course, which is more about mechanics driven gameplay design. While you are encouraged to think about your game’s story and world to imply with art and audio, we encourage you to avoid game concepts that require massive amounts of writing and branching narrative. Multi-Device Games: you are welcome and encouraged to make multiplayer games where the players are both on the same laptop (with different key-sets or XBox controllers, or even one person on keyboard and another on VIVE), but multi-device games are another major set of design topics that we are not covering here, in addition to posing significant technical challenges with Unities uNet networking system. To be clear, you CAN pitch games like these. We have had a few games that did narrative and world building and even one that succeeded in getting uNet to work by devoting one programmer to that issue alone for most of the project. BUT we encourage you to focus on single laptops for multiplayer or single player games, with focus on original interaction mechanics over narrative, as this will allow you to iterate and focus on the core concepts of this course.