6 Ways to Do More Performance Testing in Less Time

1. 6 Ways to Do More Performance Testing in Less Time NovaTAIG March 9, 2011 Steve Sturtevant, OC Systems James Pulley, Newcoe Performance Engineering

2. Presenters Steve Sturtevant Product Manager OC Systems, Inc. James Pulley CTO Newcoe Performance Engineering

3. Performance is a requirement… require it. • Most organizations don’t establish performance requirements up front. • Architects, platform engineers, developers, functional test all may have different assumptions. • Need to treat performance as you would any other non-functional requirement. • Want something that can be objectively measured. • Need to go through consensus-building process.

4. Measure twice, test once. • Test early, measure often. • Testing is always happening. • Got data? Look to your tools! • Word of caution: some developers live in Animal House. • But why? • Validate architecture early. • Low hanging fruit is fruitful. • Your schedule will thank you.

5. Accountability is king. • Get developers to buy into performance. • Create a personal connection. • Create an understanding within your metrics and methods. • Go public, point fingers. • Involvement early pays dividends later.

6. Functional testing provides performance smoke testing. • Get performance testing done before the performance testing window. • Add a couple additional elements that measure timing to functional tests. • If it doesn’t scale for one, it will never scale for many. • Send performance issues back just as you would functional defects.

7. Don’t just test, diagnose. • At the end of the day you need to fix, not find. • “More with Less,” sound familiar? • “Here’s what I think happened.” • Context is King; be factual. • Eliminate “I can’t reproduce.”

8. Don’t forget about your data. • Need data that is as real, as dirty as what you have in production. • Be prepared to deal with data issues: • Restricted data sets – data state may change. • Resource lock problems. • Testing ghosts. • Seeding larger data sets takes less time than having engineers figure out if they have a bug or testing ghost.

9. 6 Ways to Do More Performance Testing in Less Time • Performance is a requirement… require it. • Measure twice, test once. • Accountability is king. • Functional testing provides performance smoke testing. • Don’t just test, diagnose. • Don’t forget about your data.

10. Questions & Answers

11. Contact Information Steve Sturtevant James Pulley CTO Newcoe Performance Engineering 704-351-0117 jpulley@newcoe.com Product Manager OC Systems, Inc. 703-577-6633 sms@ocsystems.com