A Comparative Analysis of TestLink and Excel as Test Management Tools

1. A Comparative Analysis of TestLink and Excel as Test Management Tools Jeff Tonkovich

2. Today’s Topic • A (brief) comparative analysis of Excel and TestLink as Test Management Tools across five categories: • Setup • Usability • Reporting • Efficiency • Unique Features

3. Research Methodology • 30 minute interviews with 5 engineers with hands-on experience with both Excel and TestLink • Used standard questionnaire with each engineer plus an open-comments section • Obtained access to over 100 presentation slides for previous effort to transition test from Excel to a formal Test Management System (TMS) • TestLink was the eventual winner

4. Setup • Excel • Deployment time varies but is generally very low • May be longer if deploying via SharePoint w/ user accounts • Manual population of test cases / requirements • TestLink • Web-based w/ central DB • Requires a server with Apache, MySql, PHP & Testlink • Deployment time varies and increases with project size • Import test cases, setup user accounts, import requirements • Approx. 2 hours for medium size project

5. Usability • Excel • Used extensively in school, minimal learning-curve, extensive help and documentation available • Not ideal for multiple-user scenarios • Merging worksheets, check-in/out via SharePoint • TestLink • Perfect for collaboration between multiple testers • User Management: able to assign roles users • E.G. Test Lead, Tester, Program Manager, etc • Poor documentation, particularly for new features • Rule of thumb: • 30+ requirements: TestLink, otherwise Excel

6. Reporting • Excel • Very simple to generate test reports with graphs • Pass/fail statistics, weekly reports, etc • TestLink • No major gain over Excel for simple tasks • Built-in report generators • Auto-generated report showing requirements missing TCs • Customizable via PHP • RVTM auto-generated with click of a button!

7. Efficiency • TestLink • Rapid Test Report generation • “days” -> “hours” • Huge time savings in requirements maintenance • 4x time savings vs Excel • Rapidly generate Test Plans (TP) • TPs for different platforms, functionality (e.g. UI), etc • Test reporting less prone to human error • Bugs in Excel Macros, inconsistent reporting, forgot who tested what, etc

8. Unique Features • Excel • Simple to use but amazingly powerful • Customizable via macros, VB script and add-ons • TestLink • Email notifications • Notify test engineers which test cases / suites are assigned to them • Version control • Test Cases and requirements • Metadata tagging • Bugzilla integration

9. Summary • All interviewees unanimously agreed that TestLink is superior to Excel for Test Management • Exceptions for very small projects and projects using Excel that are almost over • Use in CS577 • Projects are small and overhead may not be worth it • Mitigate by setting up / configuring prior to semester start • Auto-generate RVTM / Traceability Matrix (portions) • Fully-functional Demo available on TestLink website

10. Thank You!