Be a SME!

1. Be a SME! Teaming with Product Verification to Gain Product Expertise in High-End Telecom/Datacom Labs By Gordon Brown

2. The Problem • In 1999, customers filed 180 problem reports, many for critical procedures, against the standard version of the key Network Administration Manual • Root cause • Writers’ lacked technical expertise because they had not performed complex procedures, which would often take several days each to successfully set up and complete without prior experience

3. The Solution • Align the Customer Documentation team by dotted line with Product Verification (aka Test) team to learn about product testing and features and to share lab equipment • Have Customer Documentation people actually do the testing themselves • Why documentation team testers? • Arms-length relationship with product • Writing and editing expertise • Share customer orientation with verification

4. The Testing Objectives • Improve the quality of the customer documentation • Use dedicated lab access to validate the key Network Administration Manual • 98% test case success rate expected • Become Subject Matter Experts • develop product, UNIX, telecom/datacom fluency • Learn skills of the verification discipline

5. The Process Customer Documentation team • Attended verification meetings • Developed test plan: 161 test cases • Booked dedicated lab shifts, followed lab guidelines • Used the latest milestone software, equipment, and network traffic generators • Peers tested the document to ensure objectivity

6. The Process (cont’d) Customer Documentation • Tracked problem reports using the Product Verification team’s tracking application (entered TCIDs, status, lab notes) Product Verification • Provided buddies to answer lab setup questions, break log jams

7. Enabling Factors • Sponsors: Verification prime, customer-facing project manager, senior documentation manager • Customer Documentation management lobbied during the previous release for an expanded testing role (quality sweep of entire release followed) • Prior influencing efforts paved the way for fuller teaming in the next release • Customer Documentation senior manager, department manager, team leaders, and information developers (writers) were all motivated to take on extra work to complete the testing objectives

8. Metrics, Metrics, Metrics! • Executed test case success rate: 98% of executable test cases • Critical TCs: 124/124 completed • Major TCs: 8/8 completed • Minor TCs: 16/16 completed • 145 successes, 3 failed, 13 non-executable • Only one outstanding problem report at the end of the exercise

9. Results • Quality improvement • Almost all problem reports closed • Many problem reports generated for tracking purposes and then closed by author • Lab access was complete and very satisfying (98% objective attained) • Significant subject matter expertise gained about product, UNIX, telecom • Verification knowledge advanced Motivation + Action = Satisfaction

10. Feedback • “This was the best example of documentation testing on a product of that level of technical complexity I’ve ever seen.” – former Senior Nortel Networks writer • “An excellent job of fostering the partnership between [the] documentation and verification team[s], which has moved forward the quality of testing and efficiency of the testing cycle.” – Verification team manager

11. Lessons Learned Subject matter expertise gained through hands-on testing • Content—the key to any document—improves dramatically if the Customer Documentation team uses labs to verify their own work • Our alignment with the Product Verification team resulted in our attaining the lab access we needed. Consequently we were able to gain the subject matter expertise we sought and significantly enhance both the quality of the documentation and the effectiveness of inter-disciplinary teaming