What can we Learn from Testing in the Games Industry?

1. What can we Learn from Testing in the Games Industry? Evolution Problems Solutions

2. Evolution Where did we come from and… where are we heading to?

3. Where did all this start? In September 1977 the first proper home console was released the Atari 2600 Not quite the first there was Pong and…

4. A bit earlier there was… 1954 A game on a scientific computer displayed on an oscillator display and it was a simple tennis simulation

5. Over the years we moved from 8 bit to now 256 bit

6. The Industry has changed At the beginning we had cartridges, Code to type in inside magazines, audio tapes and Floppy disks. Then we moved on to CD’s/DVD’s And ended up on Blue-Ray, Cartridge for Handhelds and Digital Downloads. The games industry is a constantly changing and evolving beast and you need to keep up with the newest developments at any given moment, with all the multitude of types of games, genres, hardware and business models this industry is one of the most complex and tightly knitted community. The games changed from simple Jump'n'Runs to complex virtual worlds where players are free to explore and interact which also change the development requirement radically.

7. How did gaming evolve? At the beginning there were pretty linear titles like Pacman but also first glimpses of a bold future like Pitfall which offered branched play and free exploring of the world, in this game Pitfall 2 it was a page flipping the world.

8. Then the big growth of the games business happened with the Super Nintendo and Sega Mega Drive. The sales of these consoles and titles drove a industry to boldly go beyond its know capacities and the title budgets and entertainment values grew beyond anyone's dreams and then the switch to 32 bit and 3D occurred. This brought more open worlds which were free to explore albeit limited at the beginning of the 3D games. Testing got a whole lot more complex too.

9. And then two revolutions occurred Then the big steps to conquer the movie industry happened, games became far more realistic with PS3 and Xbox 360 and the whole industry started to sell more than the movie industry per year. The games became far more complex like GTA 5 a title where you can do whatever pleases you. The world reacts to what you decide and how you perform not anymore that you have to react to a fixed world this was the big game changer along the visual candy. Testing got severely complex and it became impossible to test everything just by sheer volume of content and this mixed with the sheer infinite possibilities what players could do a new testing approach dawned.

10. Problems How the gaming sector success became its own worst nightmare which needed to be conquered

11. Complexity rising At the beginning videogames where straight forward linear games, sometimes they had an extra room to find but that was the pinnacle of complexity. These projects where similar to test like ATM’s where the customer has a similar linear experience. During the years the complexity rose and games became free roaming

12. Complexity rising

13. Free-Roaming A huge problem for testing is based on the free-roaming in games. Here you face the reality that you need to check every inch of the game against problems in 3D, which is in itself pretty improbable due to sheer size. Additionally Free-Roaming introduced the problem that the player can choose what he wants to do in whatever way it pleases him which turned testing these titles into a true nightmare for any QA professional.

14. Stakeholders and Creativity Games are an art form, they are created by teams sometimes exceeding 400 people and due their creative nature and complexity face lots of design changes when different elements of the games start to work together for the first time and do not work out as intended this accounts for roughly 60%. Also Focus group testing enables us to re-align the project better to the users taste and improve the chances of success. One factor which is attributing to at least 40% of all changes in a project are Stakeholders like Marketing and Sales Departments at the Publishers or investors from sensible requests to could you add an technical impossible part to a game or add a Barbie into a Medieval Fantasy game for 16+ males.

15. Solutions

16. Project Management changed in all areas of the games development. At the beginning all games were developed in a waterfall environment and had clear separate phases, while today's projects are often started with a vertical slice to please investors and publishers then the content and more gameplay is added on top of those this required a completely different approach. Several project management types are merged and mended to fit the needs of the project and teams which work on these titles that you don't have a clear agile or waterfall you face anything in between with a huge focus on flexibilityand adaptability. These two criteria's are also the QA’s biggest challenge, the games are so complex that they aren’t fully testable in the old sense anymore new approaches had to be taken. Procedural games are a new type of game in which are every time different for each player, which makes this type of product not 100% testable only to a certain degree which requires different testing strategies Today’s testing is based on automation testing for everything which can automated and human testing for localisation and gameplay, while the human force testing for the gameplay is focussed on the most probable gaming paths player might take not on all as some of the more modern projects would take centuries to test with thousands of testers in the old way.

17. Localization got more complex as it is not only language but full internationalization is required to adhere to cultural sensitivities and law regulations therefore companies you specialised QA companies who check for these problems during their language specific testing. The sheer size of the modern games requires armies of testers to test them and this is only possible with flexible outsourcing, but this need to be integrated into the whole strategy for the development from day one. The QA and its testing capacities, experience and strategies became a vital part of the teams and project management a far step from its beginning as dungeon inhabitants. Additionally Free 2 Play started to grow and this involves a different whole strategy in itself as the development creates only a base game and the interaction of the players and the resulting data decides which direction the development heads as content is developed in accordance to players preferences due to the business model relying on players buying extra content not the whole product like in the traditional model.

18. Thank you! If you have question please do ask now!