Marketing & Sales Roundtable Beta Program Best Practices November 2004

1. Marketing & Sales RoundtableBeta Program Best PracticesNovember 2004

2. Agenda 11:30 – Introductions/Objectives 11:45 - Beta Program Best Practices: Presentation and Discussion 1:15 – Summary and Wrap-up

3. What Do We Mean: “Beta Test”? • Testing of a working product performed by real users in a production environment • Last test before formal product launch, production shipments begin (GA) • A test of all elements and process required for successful GA product The Beta program definition will be driven by product category and type of users – HW, SW, Services

4. Objectives for a Beta Program Product • Gain deeper understanding of how product works in a ‘real world’ environment • Stress products more completely and in different ways than in a lab setting • Fine-tune product/service functionality (tweak vs. major overhaul) • Test configuration planning, installation and support procedures and documentation (including technology and delivery partners where applicable) • Test specifications and metrics to ensure quality levels of the specifications in a range of environments (e.g. usability, reliability, performance, etc.) Marketing • Test corporate and product positioning platform, message architecture and plans for customer segment programs • Develop a systematic way for capturing customer information and input • Build key early customer relationships, generate references and referrals • Understand weaknesses in company infrastructure required to support all customers’ requirements (total product)

5. Beta Program Development • Define beta • testing • process, • participants • and • objectives • Produce • integrated • functional • plan • - Engineering • - Marketing • - Sales • - Professional • Services • - Support • - Partners • Develop criteria for beta test customers • Identify and Recruit beta customers and set expectations • Create beta program • documentation • Collect and analyze user • feedback • Identify, • prioritize issues to be resolved • Implement • beta • program • - Product • Distributed to • Beta • Users • - Survey • Vehicles in • place • - Reporting • Process • - Timelines, • Milestones • Develop integrated functional response plan • Refine beta testing process • Summary beta report doc

6. Sample Criteria for Selecting Beta Customers

7. Beta Program Best Practices: The Short List • Don’t abuse the generosity of beta customers with non customer-worthy product – develop a “phased beta approach” to address risks of putting beta product in big influential prospective customer • Choose and recruit your beta customers, users to match market entry customer segment(s) • Drive for confidentiality (NDAs, Beta agreements, etc.) • Manage the program and beta users, communicate (bi-directional) regularly (program managers in both your company and customers) • Provide a simple, easy method for beta users to give feedback, make them feel like part of the your team • Encourage beta users to communicate with each other • Respond quickly to all requests and problems • Document, document, document

8. What Motivates Beta Testers ? • Early adopter technology adopter cycle value profile • Ability to contribute to product development process, be perceived as expert • ‘Community’ participation • Exclusive access to new product/Head start on resolution of their business problem • ‘Free’ product/Incentives • Career advancement Where do You Find Them? • Sales Force/Executive Team relationships • Partners (Channel, complementary products, etc.) • Internet groups/Blogs/Bulletin Boards and Lists • “Dead prospects” • Customers’ competitors • Other? • User Groups/SIGs

9. Elements of Beta Program Research Plan • [Realistic] sampling strategy • - # of customers • - # of participants • - Functional representation • Efficient user feedback (technical and marketing) collection tools (on-line, web page) • Third party tools (e.g. web surveys - Survey Monkey, Zoomerang) for simple, measurable inputs • - In-depth, qualitative (e.g. telephone/in-person interviews) for exploration of issues/requirements and positioning strategy validation • - Forum discussions (open and hosted: web forum, conference calls, WebEx) • Surveys: design for structured analysis! • - Topics to be covered by function • - Technical and marketing metrics • - Qualitative ‘discussion’ topics

10. Beta Test Plan • Definition - Project parameters, goals, activities and participants - Internal resources required, responsibilities identified -Foundation for beta report/summary • Table of Contents - Product definition (in current state) -Target customer segment(s) – “user spec” (time, HW, etc.) - Parameters (# of participants, timeline, etc.) - Goals (UI, support requirements, testimonials, etc.) - Incentives (as appropriate to situation) - Beta team action items and responsibilities - Etc.

11. Post-Beta Test • Make customer feedback accessible, institutional customer intelligence (Analyze customer feedback in structured way and document all issues) • Prioritize issues (severity and repetition) and develop an integrated cross-functional action plan • Incorporate beta user feedback into market-driven product roadmap • Ensure all company infrastructure requirements identified have been fully addressed • Review beta program process and execution performance and develop improvement plan • Send out incentives (where appropriate) and thank individual Beta users personally (phone call, email, etc.) in a timely fashion

12. Some Tips for Managing a Beta Program • Establish well-integrated cross-functional team with representation from all stakeholders (executive, marketing, sales, engineering, support, partners, etc) • Assign program owner (generally marketing) and define clear team member roles and responsibilities • Agree cross-functionally on criteria for choosing a Beta site • Promote importance and value of beta test to all functional organizations and share the beta information with all who need it

13. Key Beta Test Mistakes • “Wrong” beta testers accepted • Recruiting family or friends skews perspective • Poorly managed beta testers and/or test data • Prohibitive time commitment, resulting in drop-outs • Viewing Beta program as revenue source • Beta test period too short or too long • Too few or too many beta users (difficult to manage) “Choosing the wrong beta testers is the best way I know to provide useless results”

14. Summary: A Successful Beta Program • Product works as advertised • Benefits are documented, validated and measured (overall, feature-specific) • Product configuration, installation and customer support processes and documentation meet customer expectations • Unanticipated or un-addressed total product requirements are identified and prioritized with level of criticality determined

15. Presenters • Rosemary Remacle, Consultant Market Focus 408-244-0412 rosemary@mktfocus.net