1 / 64

Test Design and Management in context of IFDK reference product

Test Design and Management in context of IFDK reference product. About this course material. This material if for general training for Continous Integration and unit testing Material is more supportive in class room Material will be updated during courses

chung
Download Presentation

Test Design and Management in context of IFDK reference product

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Test Design and Management in context of IFDK referenceproduct

  2. About this course material • This material if for general training for Continous Integration and unit testing • Material is more supportive in class room • Material will be updated during courses • FreeNest Portable Project Platform is used to demonstrate things only in practice. This is not limiting usage for material for other training environments (I hope ) Aboutmaterial

  3. Why we need requirements from testing point of view? ”Traditional Testing Levels” „Test Engineers Area“ Customer/Business/Stake Holder Requirements Acceptance Testing System Testing System Requirements Design Requirements Integration Testing „Developer's Area“ Component Requirements Unit Testing Implementation

  4. Testing Orientation http://en.wikipedia.org/wiki/Software_testing Black Box Testing “SYSTEM TESTING” Perspective Grey Box Testing White Box Testing “CODE LEVEL TESTING” Perspective

  5. What should be tested? Input ? Output?

  6. Component /Unit Testing Developer Implemented Class Unit Test Frame Work TestClass Class Class Class Class Test Method Call Attributes Attributes Attributes TestMethodCall Method Result Methods Methods A=1 B=2 C=Class.TestMethodCountValues(A+B) C<>3 FAIL C=3 PASS MethodCountValues( int x, int y) z=x+y+1 Return z

  7. Test Driven Development DEFINE TEST CASES FIRST!!! Developer IMPLEMENT CODE AGAINST TESTS Implemented Class Unit Test Frame Work TestClass Class Class Class Class Attributes Attributes Attributes Test Method Call Methods Methods TestMethodCall Method Result A=1 B=2 C=Class.TestMethodCountValues(A+B) C<>3 FAIL C=3 PASS MethodCountValues( int x, int y) z=x+y+1 Return z

  8. Why Integrate first? Avoid Big Bang! Web Service HW Component Data Base Component/Application 10% tested Tested Component/Application Yläotsikko

  9. Continuous Integration Continous Integration GOOGLE: Jenkins CI, Hudson, Build Servers

  10. Continuous Integration in practice Test report Build Server Version Control If Unit Tests are passed – Jenkins will build whole product package SVN GIT Mercurial Etc. Execute Unit Tests for specific components http://www.youtube.com/watch?v=Ago7hpp6xZI&feature=related Youtube Jenkins CI

  11. Integration Testing

  12. Ideal project team and unit testing Software Product Integration Integration test engineer #1 Integration test engineer #2 Test Sand Box Test Sand Box Test Sand Box Test Sand Box Developer 1 Developer 2 Developer 3 Developer 4 TESTS TESTS TESTS TESTS Implemented Software Component #3 Implemented Software Component #4 Implemented Software Component #2 Implemented Software Component #1

  13. Example of IFDK productconcept IFDK = Internal Flame Drum Kit Requirement Management

  14. Example of productarchitecture Yläotsikko

  15. IFDK System Verification and Validation IFDK Product Ideas Product VALIDATION Features Customer/Business Requirements Acceptance Testing Validation = Are we building the right product? Verification = Are we building the product right? Use Cases System Requirements System Testing Architecture& Design& Implementation Sub System Requirements Integration Testing User Storys Component Requirements Component / Unit Testing VERIFICATION Requirements VALIDATION vs VERIFICATION? Yläotsikko

  16. Requirements http://www.sysml.org/ System Engineering Software Engineering - Architecture OMT SysML UML Software Engineering - Design Code OMT UML OMT .NET, JAVA, C++ Google: SysML, UML, Systems engineering, Software Design, http://www.sysml.org/

  17. Differentaspects to product QA Tester Customer Seller Developer Yläotsikko

  18. Different requirement levels Customer/Business/Stake Holder Requirements System Requirements Scalability Stability Performance Design Requirements Security Performance Stress Component Requirements Usabilty Implementation

  19. V-Model in testing Verification and Validation Customer Requirements Acceptance Testing System Requirements System Testing Integration Testing Sub System Requirements Component Requirements Component/Unit Testing Verification = Are we building the product right? Validation = Are we building the right product? Yläotsikko

  20. What is Test Design? REQ-X REQ IDEAL Requirements help Design REQ-O REQ-Z REQ-Y Yläotsikko

  21. Simple Requirement Management Process Problem Domain Solution Domain Feature X * n Vision of product Defines Functional Requirements Non-Functional Requirements Test Case Use Cases Design documents & implementation User Storys Customer/Marketing/business FEATURE VISION/NEED/PROPOSAL YOU! Solution Proposal Yläotsikko

  22. You have vision of productWhat means testing in brief? Ready to test Product Design & Implementation Testing & Quality Assurance Features Test Case Ready to Deliver Test Case Use Cases Not ready to deliver Can we deliver Product User Storys Test Case ? Customer Test Case Requirements Yläotsikko

  23. Ideal project team and unit testing Software Product Integration Integration test engineer #1 Integration test engineer #2 Test Sand Box Test Sand Box Test Sand Box Test Sand Box Developer 1 Developer 2 Developer 3 Developer 4 TESTS TESTS TESTS TESTS Implemented Software Component #3 Implemented Software Component #4 Implemented Software Component #2 Implemented Software Component #1

  24. Test Driven Development TDD Define Tests Implement Code

  25. Black Box vs White Box Testing ? ? Black Box Testing for selected component Unit Testing is White Box testing Yläotsikko

  26. Component /Unit Testing Result Unit Test Frame Work Class Class Class Class Class Class TestClass Class Attributes Attributes Attributes Attributes Methods Methods Methods Methods Yläotsikko

  27. Component Testing Product VALIDATION Customer/Business Requirements Acceptance Testing System Requirements Architecture& Design& Implementation System Testing Sub System Requirements Integration Testing Component Requirements Component / Unit Testing VERIFICATION Yläotsikko

  28. How to test? What should be tested? How ?

  29. How To Test ? What should be tested? How ? http://prosentti.vero.fi/veropros_tietojen_syotto2011.asp

  30. How to test? What should be tested? How ?

  31. How to Test? What should be tested? How ?

  32. Large System Testing DB DB What should be tested? How ? Application Application Application DB DB

  33. (Sub) Small System Testing Appcliation DB What should be tested? How ?

  34. Application Functional testing Application What should be tested? How ?

  35. Application code level testing

  36. How to create Test Case??? • Check also..... • Correct functionality path • Miss-usage of functionality • Boundary Check • Functional Test Case: • Verify functionality of XXXX • Requirement • Use Case • Feature • User Story Write a Case Functional? Which Type? Create a Test Case! Your Sources For Test Case Regression Test Case?? Non-Functional? • Customer's Idea • Brainstorm • Intitution • Exploratory • Non-Functional Test Cases • Verify Stability of XXXX • Verify Performance of XXX • Verify Security of XXXX • Verify Usability of XXXX • Verify Scalability of XXXX • etc... • Check also..... • Check Possiblity to automated testing? Write a Case Yläotsikko

  37. Card Game Functional Requirement NON FUNCTIONAL Non-Functional Requirement Non-Functional Requirement Non-Functional Requirement MAINTENANCE SECURITY RECOVERY? LOAD FUNCTION Non-Functional Requirement Agile Epic / Theme NON FUNCTIONAL Non-Functional Requirement Non-Functional Requirement Main { …. } ? SAFETY IMPLEMENTATION SCALING PERFORMANCE?

  38. SCALING USABILITY Stake Holders? Customer / User ? Business Request ? User Story ? User Story Addition + NON FUNCTIONAL Traditional Feature NON FUNCTIONAL Use Case ? UC Acceptance Criteria? RECOVERY? PERFORMANCE? SCENARIO ?

  39. Product Idea! $$$ • Cat Safehouse with web cam support • Cat’s owner can “communicate” with cat using “skype” alike technology. Bring Cat to safe house Web portal login CAT HOUSE Service 30€/day Food, drinks and internet connection included Live Cam feed Get Cat back home

  40. Heading • Sedposuereinterdumsem. • Quisqueligula erosullamcorperquis, laciniaquisfacilisissedsapien. • Maurisvariusdiam vitae arcu. Sedarculectusauctor vitae, consectetuer et venenatisegetvelit. • Sedaugueorci, laciniaeutincidunt et eleifendneclacus. Yläotsikko

  41. Component Design Class Class Class Class Attributes Attributes Methods Methods Yläotsikko

  42. Calory Counter: Player can measure calories during training session. This can be seen as exercise result in web service eg. Facebook application Feature example 1 (Invented on course 2009-2010) Feature X * n Energy usage Yläotsikko

  43. SW Development Process (Agile) Product Backlog Task4 User Story X Task5 Task6 Task1 Task7 User Story Y Task2 Task8 Task3 User Story Z Task9 User Story Z Sprint Sprint Sprint Sprint Design Implementation Verification Design Implementation Verification Design Implementation Verification Design Implementation Verification Yläotsikko

  44. Testing according ISEB standard Functional Testing Testing based on an analysis of the specification of the functionality of a component or system. See also black box testing.) Non-Functional Testing Testing the attributes of a component or system that do not relate to functionality, e.g. scalabilty, stability, reliability, efficiency, usability, maintainability and portability. http://www.bcs.org/upload/pdf/glossary-current.pdf Yläotsikko

  45. Notes • We need to capsule all projects as sub projects, which have more independence • Project Teams has to have own test engineer –> Integration Test Engineer • this lead’s to better defect prevention • Integration Test Engineer is part of project team, part of team at start of project • Test Engineer provides valuable information for design and enables better testability for project product • Define new roles for validation engineers • Check next page • Lower level for test automation • Enables regression test (method to block new implementation defects) • Support for executing more complex test scenarios • Continuous test execution (stability, performance gain etc) Yläotsikko

  46. Testing Level descriptions • Test Level1 – Component/Unit testing • Component/Class level unit testing (eg. xUnit framework) • Test Level2 - Integration Testing (Feature, Application) • Integration test for feature/application with “stub” interface components • Functionality SW Testing • Non-Functional SW Testing according needs (eg. feature/application specific performance, stability) • Test Level3 – Regression Testing (Target HW+Android platform, tool, terminal etc.) • Functional SW testing for all possible features • Non-Functional SW testing according needs • Non-Functional HW Testing according needs • Test Level4 – System Integration Testing (Target Platform, Tool, Terminal etc) • Functional SW testing for all new features • Non-Functional SW Performance, Stability, Scalability • Test Level5 – Acceptance Testing • Functional/Non-Functional test according customer requirements • HWT1 – Hardware Testing • HW Integration test with limited environment. • Conducted performance verified Yläotsikko

  47. Testing Levels (with TAF = test automation framework) IFDK Release v1.0 Acceptance Testing (ISEB) System Testing (ISEB) TAF enables TAF TAF Regression Testing (ISEB) System Integration Integration Testing (ISEB) TAF Supports TAF Project Component/ Unit Testing (ISEB) Feature Project 2 Feature Project 1 Verification and Validation Yläotsikko

  48. Refrerences & Links http://www.rbcs-us.com/images/documents/The-ISTQB-Advanced-Syllabus.pdf Yläotsikko

More Related