1 / 13

Seeing is Believing: Using Streaming Video of Usability Testing for Realtime Reality Checks

Seeing is Believing: Using Streaming Video of Usability Testing for Realtime Reality Checks. Katherine Isbister, Karl Steiner, Chris Cross 1 March 2011. The Challenge: Making an Impact on Design with Testing. Not all designers can attend tests in person

serge
Download Presentation

Seeing is Believing: Using Streaming Video of Usability Testing for Realtime Reality Checks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Seeing is Believing: Using Streaming Video of Usability Testing for Realtime Reality Checks Katherine Isbister, Karl Steiner, Chris Cross 1 March 2011

  2. The Challenge: Making an Impact on Design with Testing • Not all designers can attend tests in person • Written reports and video clips reframe and summarize, miss many nuances. • Slogging through video records after the fact is time consuming (and likely rare).

  3. The Context • AAA game, late-stage development, need for regular play testing cycles, team very busy. • Decision to test off-site at NYU-Poly’s Social Game Lab. • Streaming as an easy way to ‘drop in on’ testing, for the studio team and for corporate folks (e.g. THQ’s Karl Steiner).

  4. Technical Set-up • Combined hardware/software solution • Hardware: • Mac Mini, 4-channel security camera DVR with 1-2 cameras attached • Canopus ADVC-110 Advanced Digital Video Converter • M-Audio MobilePre USB Audio pre-amp • Headset with stereo headphones for player • Laptop running Skype, other smaller components

  5. Technical Set-up • Software: • Quicktime Broadcaster with a Quicktime streaming server. • Broadcasts and records to server at the same time.

  6. Issues that came up • HD output requirement for game output threw us for a loop early on with cabling and splitting. • Network lag made using the stream as the moderator infeasible (parallel Skype window work-around) • Studio needed to open network to allow streaming • Streaming required broadband connect. • Quality audio blending (player, game, moderator) was challenging, because of the need for 2-way communication as well as streaming.

  7. Sample Footage

  8. How It Worked • Bi-weekly testing every Thursday and Friday afternoon. • Cycled through most levels (SP and MP) during the testing period. • At least one designer was always on hand for the testing. • Others tuned in via the stream.

  9. How Streaming is being Scaled at THQ • <Karl—bullet points here?>

  10. Conclusions • Though sometimes glitchy, streaming play tests seemed to add real value for the design team. • The ‘tuning in live’ factor was especially useful. • The technique may help your team gain deeper empathy and insight into the player’s experience of your game, and move the impact of play testing more deeply into intuitive micro-level design decisions that add up to an excellent game.

  11. Thanks! • isbister@poly.edu • <karl—your email address here?> • Many thanks to Rahul Rao, Ashwin Ramesh, and John Mahlman for their hard work on this at NYU-Poly.

  12. Details on Audio Set-up • Had to stream blended sound, but also, allow 2-way communication between moderator and player. • Complex path led to difficulties in volume adjustments. Was/is a work in progress. • (Audio from console and player headset to laptop running Skype via a splitter. Audio from laptop run into another audio splitter which sends the audio to the headphones and the M-Audio USB Preamp. The preamp allows the use of multiple inputs to be sent into the Mac Mini via USB. All audio (player and overseer) is recorded and streamed at the same time. )

More Related