10 th T3UC 2011Conference, 7-9 June 2011, Bled, Slovenia Invited Talk: Future Testing Challenges

1. www.isqtinternational.com 10th T3UC 2011Conference,7-9 June 2011, Bled, SloveniaInvited Talk:Future Testing Challenges Arunkumar Khannur ISQT Process and Consulting Services Pvt Ltd #732, I Floor, 12th Main, III Block, Rajajinagar, BANGALORE- 560 010 INDIA Email: khannur@isqtinternational.com www.isqtinternational.com Ph: +91-80-23145564/65

2. www.isqtinternational.com Points that We Discuss • Initial remarks • Software testing challenges • What is real and what is represented always differ • Addressing an unreliability of our perceptions • Where are we heading? • Top 3 test challenges, risks, and solutions

3. www.isqtinternational.com Initial Remarks • Software Testing is a process of measuring quality of a software • Software Testing has many challenges

4. www.isqtinternational.com Software Testing Challenges • Quality is relative • Two types of quality • Primary or Intrinsic • Ex: Functional aspects • Secondary or Extrinsic • Ex: Non-functional aspects • Unrealistic Requirements Specifications

5. www.isqtinternational.com Software Testing Challenges • Increasing Technical Complexity • Increasing Managerial Complexity • Test related Standards and Frameworks are yet to mature • Immaturity in Book of Knowledge

6. www.isqtinternational.com What is Real and What is Represented always Differ • what is real and what is represented always differ.  • Requirements specifications is a "representational" perceptual model that is built based on inner "ideas", "impressions' or "sense data" of an observer (requirements study team member) and his inferences • Thus, most of the errors in software have origin in requirements

7. www.isqtinternational.com How can We have Better Representation of Requirements? • If we find a way to directly establish a link between the observer's inner world and external object, we can have better representation of requirements. • However, major hurdle to achieve this is unreliability of our perceptions

8. www.isqtinternational.com Addressing an Unreliability of our Perceptions • To reduce the gap between perceptual model of our inner ideas and outer objects , we need to understand that there are two types of qualities, namely, • Primary Quality (Absolute Quality) and • Secondary Quality (Relative Quality).

9. www.isqtinternational.com Primary Vs Secondary Quality • What is real and what is represented always differ •  If we find a way to directly establish a link between the observer's inner world and external object, we can have better representation of requirements. • However, major hurdle to achieve this is unreliability of our perceptions.

10. www.isqtinternational.com Secondary Quality • Secondary Quality • Qualities of any object that are extrinsic (not intrinsic) and vary based on time and conditions and as such which define relative quality • Our ideas that we develop in our mind  on secondary qualities never resemble the the physical object itself

11. www.isqtinternational.com Example: Secondary Quality • The colour of the User Interface is not a property of the screen itself but a product of • the interaction of various factors, including certain physical attributes of the screen such as power supply, resolution, • the peculiarities of our own sensory system; and • the environmental conditions prevailing at the time of the observation

12. www.isqtinternational.com Primary Quality • Primary Quality • Qualities of any object that are intrinsic and do not vary based on time, perception and conditions and as such which define absolute quality. • our ideas that we develop in our mind  on primary qualities closely resemble the physical object itself • Thus, primary qualities of physical objects define absolute quality • Primary qualities also help us • in explaining and also, • developing an experience of the secondary qualities

13. www.isqtinternational.com Example: Primary Quality • Example • User Interface screen has certain true properties which are intrinsic, such as its size and shape, which do not depend on the conditions under which the screen is observed or on the existence of the viewer

14. www.isqtinternational.com Primary Quality and Requirements Specification • While capturing requirements always think on primary qualities of clients wants and needs. • If you are able to identify such primary qualities, then you can have concrete requirements that are beyond scepticism. • Such requirements which can be represented using primary qualities are implementable and measurable. • For example, requirements like accuracy of numbers, length of any text field, number of permissible users, number of transactions that need to be supported by the system etc are primary qualities

15. www.isqtinternational.com Secondary Quality and Requirements Specification • Secondary qualities of requirements can not be concrete and as such they are the basis for skepticism. • Secondary qualities can not be implemented to perfection and also, can not be measured. • For example, requirements like system shall be user friendly, system shall have recoverability feature, user interfaces shall be pleasing etc are secondary qualities. • Thus, while arriving at requirements specification if we focus on primary qualities  then our requirements will be concrete. Else requirements will be representation full of scepticism.

16. www.isqtinternational.com Focus during Requirements Specification • Thus, while arriving at requirements specification if we focus on primary qualities  then our requirements will be concrete. • Else requirements representation will be full of scepticism

17. www.isqtinternational.com IncreasingTechnical Complexity

18. www.isqtinternational.com Increasing in Technical Complexity

19. www.isqtinternational.com Where are We Heading?

20. www.isqtinternational.com Point 1: From QC to QA to QE

21. www.isqtinternational.com Point 2: Transformation in Approach • At Present • Process Centric (Exploitation) • Plan and Design Centric • Focus is on Test Coverage (Efficiency Centric) • Test Acceptance Criteria is “Zero Defect” centric • Focus is on Return on Investment (RoI) • Future • Exploratory, Lean, Agile, and Risk based • Strategy and Architecture • Focus will be on Reducing Risks (Effective Centric) • Test Acceptance Criteria will be “Manageable Risk” centric • Focus is on Return on Expectation(RoE)

22. www.isqtinternational.com Characteristics of Exploration and Exploitation

23. www.isqtinternational.com Effort towards Value Creation Intuitive Thinking Analytical Thinking Creativity and Innovation Declaring Truth and Certainties Design Thinking

24. www.isqtinternational.com Point 3: Effective STLC • At Present: STLC • Plan and Design Centric • Scoping • Planning • Designing • Execution • Defect Management • Closure • Future: STLC • Strategy and Architecture Centric • Test Analysis • Quality Profiling • Modeling • Strategy • Test Approach Selection • Risk Management • Closure

25. Arriving at Test Strategy: Preparatory Work Requirements Specification Identify Test Conditions Requirements Review Identify Test Basis Test Requirements Elicitation Identify Test Oracles Feasibility Analysis Understand Context Defining Test Procedures Quality Profiling Test Analysis Define Test Scope TEST STRATEGY

26. www.isqtinternational.com Point 4: Dominance of Model based Approaches • Modeling of Requirements • Visual Modeling • Quality Profiling • Model based Testing Techniques • Use-case based • State Transition Diagram based • Decision Table based • Etc

27. Accuracy Compliance Interoperability Security Suitability Fault tolerance Maturity Functionality Recoverability Reliability Resource behavior Time behavior Efficiency Learnability Usability Operability Understandability Portability Adaptability Maintainability Installability Replaceability Analysability Changeability Stability Testability Representing Expected Quality Profile from Requirements Requirements Business Technical Functional Non- Functional

28. www.isqtinternational.com Point 5: Transformation in Test Automation • Model based Test Automation Tools bacome more and more popular

29. www.isqtinternational.com Point 6: Non-functionality Testing • Non-functional testing draws more and more attention • Quite difficult to automate • Major reason for Customer dis-satisfaction • Solution is in Quality Engineering

30. www.isqtinternational.com Point 7: Evolution of Standards and Frameworks • Standards and frameworks receive major attention • Book of Knowledge • Test Maturity Models • Product Quality Standards • Governance-Risk-Compliance related standards • etc

31. www.isqtinternational.com Top 3 Test Challenges, Risks, and Solutions

32. www.isqtinternational.com Top 3 Test Challenges, Risks, and Solutions • Challenge 1: • “Delivery Deadline” Syndrome will continue • Risks • Lesser test effectiveness • More stress on testing team • Solution • Model Based Testing will be effective • Optimum test cases • Increased test effectiveness • Strategy to select few test conditions and lesser rounds of testing

33. www.isqtinternational.com Top 3 Test Challenges, Risks, and Solutions • Challenge 2: • Using CMMI like process for testing • Risks • “Process Overhead” • “Too many metrics with lesser relevance” • Solution • Switch over to TMM based processes • Focus on risks than on metrics • Stability

34. www.isqtinternational.com Top 3 Test Challenges, Risks, and Solutions • Challenge 3: • Non-functional testing will be the major area of focus • Risks • Not enough skilled professionals • Attrition • Solution • Continuous training to prepare a pool of non-functional professional • Back-up for every role

35. www.isqtinternational.com Questions ?

36. www.isqtinternational.com THANK YOU