TTCN3 as Another Brick in the Wall of Test Automation

1. TTCN3as Another Brick in the Wall of Test Automation Authors: Gergely Horváth (gergely.horvath@nsn.com) Róbert Vastag (robert.vastag@nsn.com) TTCN-3 User Conference 2011 7 – 9 June 2011 - Bled, Slovenia

2. Some Facts About Nokia Siemens Networks • Joint Venture of Nokia and Siemens • Started operations on April 1, 2007 • €12.7 billion net sales in 2010 • 120+ years of telecom experience • 65,000+ employees • 45,000 service professionals • (including externals) • 75 of top 100 operators worldwide • 150 countries • 3 billion mobile subscribers and ¼ of • world’s voice households served

3. Agenda User Plane Routing Development (what we do) Agile and TTCN3 (how we do it) Test Automation with TTCN3 (how TTCN3 is integrated into it) TTCN3 Core and Fast Test Case Creation (how it works exactly)

4. User Plane Routing Development Control System Under Test: Main focus is on user plane routing functionalities Data H.248 H.248 USER PLANE TTCN3 executable: Connected with the SUT through H.248 interface, simulating MGW functionalities RADIO ACCESS • Each interface of the SUT is captured • Radio access is simulated by the traffic generator • MGWs are simulated by an executable built from TTCN3 code that is specific to each test case

5. User Plane Routing Development (what we do) Agile and TTCN3 (how we do it) Test Automation with TTCN3 (how TTCN3 is integrated into it) TTCN3 Core and Fast Test Case Creation (how it works exactly)

6. Agile and TTCN3 • Automated unit tests • Scheduled and automated • SW builds • Automated test execution • environmentforregression • (set of functional tests) Continuous Integration Feature Requirements Iterative Feature Deployment Agile Fast Feedback • New feature increment implemented in every iteration • Scrum meetings • Incremental test case design(TTCN3) • Fast module test environment • for "instant" checking of the • newcode • Nightly functional regression • test set execution

7. Agile and TTCN3 Test Design • Developing TTCN3 core functions to followchanges in user plane routing • Creating test cases using TTCN3 • MGW simulation by TTCN3 executable • Acquiring results Implementation Test Planning Specification Unit Test Test Execution Evaluation Development & Unit Test • Evaluating TTCN3 Results • Fine tuning the test case Verification TTCN3 Core Development & Integration Test Automate Tests & Add to Regression Set Feature Requirements Feature Deployment

8. User Plane Routing Development (what we do) Agile and TTCN3 (how we do it) Test Automation with TTCN3 (how TTCN3 is integrated into it) TTCN3 Core and Fast Test Case Creation (how it works exactly)

9. Test Control Equipment Pool Test Automation with TTCN3 Test Management System Manager • Tester • Test Designer • Test Architect • TTCN3 Key User Test Automation Engine SVN Repository for Test Macros and TTCN3 Test Laboratory TTCN3 Build Server Analysis Server Pool

10. Test Laboratory Test Control Equipment Pool Test data from SVN Configuration data and command sequence LAB NETWORK Collecting execution results MGW simulation with TTCN3 SUT POOL

11. Test Automation with TTCN3 Test Control Equipment Pool Test Management System Manager • Tester • Test Designer • Test Architect • TTCN3 Key User Test Automation Engine SVN Repository for Test Macros and TTCN3 Test Laboratory TTCN3 Build Server Analysis Server Pool

12. Test Set Execution Report Resource Usage Report Test Execution Report TTCN3 Result in XML Test Automation Engine • Resource usage report • Resource reservation • Test set execution report • Test execution reports • Archivation of results • Interface to test management system • Authorized access (Tester, Test Designer, Test Architect…)

13. Test Automation with TTCN3 Test Control Equipment Pool Test Management System Manager • Tester • Test Designer • Test Architect • TTCN3 Key User Test Automation Engine • Traceability Support • Requirement FulfillmentCheck • Maintenance of Test Documents • Management Reporting • Statistics SVN Repository for Test Macros and TTCN3 Test Laboratory TTCN3 Build Server Analysis Server Pool

14. Test Automation with TTCN3 Test Control Equipment Pool Roles & Responsibilities: Tester Executes and analyzes test cases, writes incident reports Test Designer  Creates test cases according to the test plan Test Architect  Responsible for regression set creation and high level planning TTCN3 Key User  Responsible for the maintenance of the TTCN3 Core (common functions used for test case creation) Test Management System Manager • Tester • Test Designer • Test Architect • TTCN3 Key User Test Automation Engine SVN Repository for Test Macros and TTCN3 Test Laboratory TTCN3 Build Server Analysis Server Pool

15. User Plane Routing Development (what we do) Agile and TTCN3 (how we do it) Test Automation with TTCN3 (how TTCN3 is integrated into it) TTCN3 Core and Fast Test Case Creation (how it works exactly)

16. A Sample H.248 Message: TTCN3 Core System. Why? • Resource Reservation • Resource Modification • Resource Removal • Service Change • Audit H.248 messages have a difficult hierachy and convey many parameters Building a template that can match a specific message is a difficult task, even for an experienced tester. Solution: • New user plane control functionalities are • implemented in TTCN3 core tomake testers’ • lives easier. • Only high level parameters need to be given

17. TTCN3 Core System. How? Features of Test Steps: Main Parameters In Testers’ Hands • Parameterized functions that can construct templates for H.248 messages • Compliant with the H.248 standard • Each test step implements one H.248 transaction • It hides the complex internal logic from the tester + Complex TestSteps = …and Benefits of its Usage: Specific Message Template Actual H.248 Message • Only high level knowledge is necessary forTTCN3 test case creation • Makes test case creation fast and efficient • Test cases are well structured • Global modifications in the functionalities of the SUTare easy to follow by updating the TTCN3 core (TTCN3 key users’ task) Compare Pass Exact location of the failure is shown! Fail

18. TTCN3 Library Structure • Test cases of the same feature compile into the same executable • The executable has to bestarted with the test caseID as input parameter in order to execute theproper MGW simulation TTCN core Feature 1 FEATURE_1_TEST_CASE_1 FEATURE_1_TEST_CASE_2 . FEATURE_1_TEST_CASE_N Feature 2 FEATURE_2_TEST_CASE_1 FEATURE_2_TEST_CASE_2 . FEATURE_2_TEST_CASE_N Feature N

19. TTCN3 Test Case Structure module FEATURE_1_TEST_CASE_1 { import from TTCN_CORE all; testcase tc_FEATURE_1_TEST_CASE_1() runs on CORE_TEST_COMPONENT system TEST_SYSTEM { SET_CONNECTIONS(); TEST_COMPONENT.start(TEST_SEQUENCE_1()); } TEST_SEQUENCE_1() runs on TEST_COMPONENT { Init(); TEST_STEP_1(PARAM1,PARAM2); TEST_STEP_2(PARAM1,PARAM2,PARAM3,PARAM4); TEST_STEP_1(PARAM5,PARAM6); TEST_STEP_3(PARAM5,PARAM6,PARAM1,PARAM2); RESOURCE_REMOVAL; } } Test step definitions Module Connectivity settings Actual Test Case Behaviour

20. Conclusion • The TTCN3 core system makes it easier to cope with frequent • changes in the software that we are testing • It is thoroughly integrated with the automated test environment • that makes its usage smooth and locating faults easy • These characteristics aid it in supporting the agile development • This way we can ensure efficient testing procedures and provide • highquality software products

21. Thank you!Your questions are welcome!