Mobile Application Testing

1. White Paper Mobile Application Testing

2. Index Introduction 3 Understanding Mobile Application Testing 4 Challenges faced while testing mobile applications 6 Why are we amongst the best in mobile application testing? 7 Methodology 8 Automation Testing Strategy 10 Conclusion 11

3. Introduction With the advent of smartphones, a mobile phone has evolved from being just a communication device to a highly sophisticated business utility equipment. With smartphones no longer remaining a conspicuous consumption, they have become an efficient tool for handling finances, managing mails & agendas, and executing other business functions. Statistics highlight that 1.6 billion smart phones are sold at a rate of 1.8 to 2 million devices a day since 2007, and 1.5 billion smartphones will be shipped in 2016 – a 5.7 percent increase from 2015 – and will reach 1.92 billion by 2020. Mobile applications are the sole reason for the rise in popularity of smartphones. The ease and convenience of mobile applications usage has created a huge dependency on it. Over the years, the number and variety of consumer and enterprise mobile applications has grown phenomenally. Considering the abundance of mobile applications and the low cost of purchasing one, uninstalling a non-responsive application is just a touch away. This puts a lot of pressure on organizations developing mobile applications. They must adhere to stringent quality control measures to ensure they do not lose out on the business significance. The development of mobile application is a very complex process and hence the possibility of bugs and errors cannot be ruled out. Testing of mobile applications becomes a pre-launch mandate for organizations to ensure that they stay ahead of the competition. Another aspect highlighting the importance of mobile testing is the variety of applications available for the public. Hence an error-prone 'app' is very unlikely to be forgiven by people and soon the app would be on its way out (of business). 3

4. Understanding Mobile Application Testing Testing is an activity of verifying whether a system or a product meets customers’ requirements. While functional testingverifies the intended functionality of the system/product, non-functional testing however verifies how the system operates and the parameters which are not related to specific function or action. Mobile application testing verifies installation, compatibility, responsiveness, consistency, and layout of the application in addition to functional and non-functional aspect of it. Wikipedia defines the term Mobile Application Testing as, ‘’Mobile application testing is a process by which application software developed for hand held mobile devices is tested for its functionality, usability and consistency. Mobile applications either come pre-installed or can be installed from mobile software distribution platforms.’’ Mobile application testing is not a conventional desktop testing and has to be device and platform agnostic. Due to varied OS platforms and browsers, numerous compatibility issues can be identified in addition to the functional and non-functional issues. So there is a need for focused application testing on mobile devices. Mobile applications are classified into Native, Web and Hybrid, the testing strategy for each would differ as per the nature of the application. Native applications are built for the specific device and its operating system, for example an application built for android device may work only on android devices. Mobile testing for native app ensures smooth download and installation of the app. It will also cover backward compatibility. 4

5. Web applications are for different platforms and are accessible across the internet without being explicitly installed on any specific device or platform. Web application can be accessible from anywhere so testing strategy would not require downloading and installation of the application. Hybrid applications are a combination of both native and web applications. Most crucial and complex testing would be testing of hybrid applications, which can be executed anywhere and on any device. Checking compatibility on each platform with intended functionality is cumbersome. Thus in all to have robust testing strategy for the mobile application is important. Developing testing strategy in end user’s perspective is very important. While using application the common challenges faced by mobile application users are: ∠ The application is unresponsive for long time ∠ The application crashes ∠ The application consumes significant bandwidth ∠ The impending security concerns 5

6. Challenges faced while testing mobile applications The multimodal platforms, operating systems, and device types create a plethora of challenges for a mobile application. Some common challenges include: Range of Mobile Devices There are innumerable types of mobile devices available in the market. Consumers have multiple selection choices – ranging from handsets to smart phones, tabs, iPads, wearable tech etc. This creates a huge diversity of environments that your mobile app faces. Mobile testing in this wide range of devices and environments is challenging, especially when the size of the device plays a major role with over 130 resolutions. Device Fragmentation & Various OS Platforms As per the customer’s requirement, mobile apps can be installed on any operating systems like iOS, Android, Blackberry, Windows etc. Real world testing on these many OS is challenging, as mobile apps behave differently on each OS. Mobile application can also be deployed on any environment or device that encompasses multiple versions of an operating system such as iOS 8, iOS 7.X, BB 10.X, Windows 8.X, windows 7.X, 2. Mobile App Types Mobile apps are not just limited to being native, it has evolved to being web and hybrid app. Mobile testing of each type of app is different. The end users always require the mobile app to be supported on all kind of environments. Numerous Test Interfaces Exhaustive testing on real device is not possible all the time with numerous test interfaces. Emulators and Simulators are important tools and they are used to verify general functionality and perform regular regression testing. Since the testing in virtual environment will not give the real world test results, so their scope is very limited. With the huge quantum of mobile applications competing to capture the biggest market share, there is a greater need for a robust testing methodology. Thus to say, ‘Mobile Testing is important’ is an understatement! 6

7. Why are we amongst the best in mobile application testing? Jade Global has created its own Mobile Testing Lab with range of devices for real world testing. We understand mobile apps and its intended use, and then target the most important devices. It gives confidence and maximum testing coverage. Jade Global adopts a holistic test strategy encompassing emulators, real devices and Cloud to reducing cost and manual effort. Having the right mix of emulators, real devices and Cloud apps, Jade Global has adopted a unique testing strategy for maximum testing coverage. It covers four main aspects while developing mobile application testing strategy: End User perspective Mobile Application Testing Configuration and Installation Perspective Functional and Non-functional perspective Compatibility perspective Special focus is given to critical cases like poor network, low signal strength, unexpected impacts on memory, low battery power, specific constraints as per device, and recovery after fatal error. 7

8. Methodology Real Devices Real world testing is required with different screen sizes to check the downloading and installation process for the native and hybrid app, checking responsive screen with change in resolutions, configurations and technical compatibility. It covers functional and usability testing, and validation of all tests in testing lab before the app goes live. Emulators Requirement keeps changing till the time app ships. So it’s not feasible to repeat real world testing after every requirement change or implementation. Jade Global uses emulators to identify bugs and functional problems. To consider the impact of security and the network we then move the testing to real device. Unlike conventional scenarios, it is important to anticipate about the unexpected behavior of the mobile app in certain situations, and check the performance and other conditions end users may often run into. Jade Global testing team considers following key considerations while testing using emulators: ∠ Bad Network ∠ Low signal Strength ∠ Unexpected impacts on memory ∠ Low battery power ∠ Specific constraints as per device ∠ Recovery after fatal error Cloud Solution Many times the budgeting is low with strict timelines, with high costing per test or large number of test cases. In such scenarios, Jade Global testing team adopts Cloud testing solution. It has unlimited storage, quick availability of the infrastructure with scalability, flexibility and availability of distributed testing environment to reduce the execution time of testing of large applications and this leads to cost-effective solutions. 8

9. Performance testing In addition to conventional testing it is important we test the performance and security of mobile apps. This is quite different from performance testing of web or desktop application. With more and more users moving to smartphones the margin of performance is getting slimmer and hence performance testing becomes an integral part of mobile app testing. Using emulators, same test can be run on thousands of devices to check the performance of the application, since it also reduces the testing cost and time. This cloud testing solution can also be used to web based and hybrid apps. It is difficult to test the performance of a mobile app in real world using actual devices. Depending on customer specific need performance testing on real devices can be achieved. Security testing With people executing huge monetary transactions over their smart phones, the hackers also start targeting mobile apps. To avoid the penetration of hackers to the mobile apps - security testing is essential. Following techniques are used for mobile security testing: 1. Penetration testing: This test is done with the intention of finding security weakness by penetrating dummy data into the system 2. Password cracking: This is the process of recovering password from data which is stored in the system 3. SQL injection: Inserting malicious SQL statements into an entry field for execution 4. Cross site scripting: Inserting malicious script 9

10. Automation Testing Strategy Jade Global has devised a solution that has tremendously reduced the manual testing efforts with approximately 70-80% ROI. We have delivered Regression, Smoke and Functional mobile automation suite to our clients. An automation test suite is discovered from the manual test cases, and a strategy is devised to automate required test cases. Around 70% to 90% manual test cases will be automated to ensure a maximized ROI. Depending on the requirement and application under test, a customized automation framework is created that is device specific. The reporting and quality metrics are also customized as per the requirement, and updated on the client’s Knowledge Base along with other process improvements. 10

11. Conclusion Devising a testing strategy goes a long way in minimizing bugs and increasing efficiency of the mobile application. The testing strategy has to ensure 100% test coverage with the best-fit tools and simulators. With new technologies introducing new mobile ecosystems, it is also necessary to manage a proper Knowledge Base. Though mobile application testing has innumerable challenges, the key is to devise an end-to-end testing strategy that blends manual and automated testing. The testing methodology should also evolve with the evolving mobile ecosystems. 11

12. About the author “ Snehal Landge is an Associate Manager – QA with Jade Global, having 13 years of experience, including Snehal Landge Associate Manager - QA leading teams and developing testing expertise in different areas (mobile application, functional, security, performance, automation). She has worked on mobile application testing projects for more than 6 years. Snehal has extensive experience in manual and automation testing using the tools like Quality center, JIRA, Selenium, Ap -Test manager on multiple OS. ” About Jade Global, Inc. Jade Global is a premier Advisory, Integration, Testing, Cloud & Consulting Services, Business Solutions and IT outsourcing company headquartered in San Jose, California with offices in San Diego, Atlanta, Reading, Pune, Noida and Hyderabad. With more than 600 employees worldwide, Jade Global is a trusted partner of choice to its clients. Jade’s global team of experts successfully deliver business and technology solutions across diverse industries to drive its customers' long term growth and success.