1 / 16

MetaAutomation:

MetaAutomation:. A More Effective Way to Measure and Promote Software Quality. MetaAutomation. Taken individually, these are not new ideas. What’s new: the Meme. MetaAutomation.

carver
Download Presentation

MetaAutomation:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MetaAutomation: A More Effective Way to Measure and Promote Software Quality

  2. MetaAutomation • Taken individually, these are not new ideas • What’s new: the Meme

  3. MetaAutomation Metaautomation is a meme of well-known practices and interrelated software technologies, used to communicate, plan, and execute the scaling-up of automation in strength and effectiveness, and integrating software quality more effectively into the SDLC.

  4. … it’s about • Improving test automation • Improving performance of Test team • Getting better value • Actionable failures • Seeing patterns of behavior • Smart retry • Automatic triage

  5. MetaAutomation • First Order First-order metaautomation describes technologies applied at automation runtime. • Second Order Second-order metaautomation includes techniques applied to artifacts of one or more automated tests at some time after a test run is complete.

  6. First-Order MetaAutomation • Add more verification points • Failures as actionable as possible

  7. Second-Order MetaAutomation • Save those artifacts! • Structure of Data is important

  8. Test Automation Failure Samples TimeoutException, and stack trace. Error: No response from server !!! … and, a stack trace. Error: The mail server timed out after ‘10000’ MS … and, a stack trace.

  9. Test Automation Failure Samples Error: The mail server ‘emailsRus.com’ timed out after ‘10000’ MS … and, a stack trace. Error: On checking for new emails, The mail server ‘emailsRus.com’ timed out after ‘10000’ MS … and, a stack trace.

  10. Test Automation Failure Samples <TestFailure> <DateTime>.. <MethodName>RefreshClientView.. <Parameters> <Parameter name=‘URN’>.. … </Parameters> <State>Checking for new emails.. <StackTrace> <StackElement>..

  11. Second-Order MetaAutomation • Automated triage! • Smart retry!

  12. Related Issues on Automation Quality • For GUI, the need for manual testing never goes away completely. • Business Logic testing • Atomic tests • Write automation in same language as product

  13. Related Issues on Automation Quality • Custom Exceptions • Wrapping up context • Logs vs. Exceptions

  14. Managing it All… • Need new Metrics! • Have testers work closely with devs • System for code reviews

  15. Quality is Always Becoming More Important! • Give testers respect and resources… those guys have a tough job!

  16. Blog • http://metaautomation.blogspot.com/ • Contact • Matt Griscom • matt@wisewillow.org • http://www.linkedin.com/in/matthewgriscom • Slides • On SeaSPIN site

More Related