Automated Concolic Testing of Smartphone Apps

Automated Concolic Testing of Smartphone Apps SaswatAnand Stanford Univ. MayurNaik Georgia Tech. Hongseok Yang Univ. of Oxford Mary Jean Harrold Georgia Tech.

Motivation

Motivation Problems with Smartphone Apps

Problem Automatically generate test inputs for bounded exhaustive testing of smartphone apps

Test Inputs for Apps Whole-program testing Test input is a sequence of events e1, e2…, en Types of events: a tap on the screen, change in geo-location, arrival of a SMS message, etc.

Bounded Exhaustive Testing of Apps S, the set of all event sequences* s.t. each sequence takes a unique path Set of covered branches Goal: cover these *of bounded-length

Two subproblems Generate individual events Generate sequences of events

Generating Individual Events • An event is associated with data • X & Y coordinates of a tap event • geo-location of a change-in-geo-location event • content of an incoming SMS event • etc. • Data determine which program path is taken Challenge: Generate the “right” data for events

Example: Music Player App Play Pause Rewind Skip Stop Eject

Example: Music Player App public void onClick(View target) { if (target == play) startService(new Intent(ACTION_PLAY));else if (target == pause) startService(new Intent(ACTION_PAUSE)); else if (target == skip) startService(new Intent(ACTION_SKIP)); else if (target == rewind) startService(new Intent(ACTION_REWIND)); else if (target == stop) startService(new Intent(ACTION_STOP)); else if (target == eject) showUrlDialog();} tap(136, 351)

Generating Individual Events Existing alternatives • Random Testing • Cannot perform systematic/exhaustive testing • Platform-specific tools (e.g., hierarchy viewer in Android) • Limited to GUI Events • Cannot handle third-party GUI widgets

Generating Individual Events Our solution Use concolic execution to generate data associated with events

Generating Individual Tap Events tap(int x, int y){ 1 if (x>2 && x<4){2 if (y>1 && y<3) 3 W1_clicked();4 else5 W2_clicked();6 }else7 W3_clicked(); } x>2 && x<4 1 F T y>1 && y<3 2 7 T F 5 3

Generating Individual Tap Events x>2 && x<4 1 F T y>1 && y<3 2 7 T F 5 3 tap(1, 5)

Generating Individual Tap Events x>2 && x<4 1 F T y>1 && y<3 2 7 T F 5 3 (x>2 && x<4) tap(1, 5) tap(3, 5) F1 !(x>2 && x<4) W3_clicked()

Generating Individual Tap Events x>2 && x<4 1 F T y>1 && y<3 2 7 T F 5 3 (x>2 && x<4)(y>1 && y<3) (x>2 && x<4) T1 (x>2 && x<4)F2 !(y>1 && y<3) W2_clicked() tap(1, 5) tap(3, 5) tap(3, 2)

Generating Individual Tap Events x>2 && x<4 1 F T y>1 && y<3 2 7 T F 5 3 (x>2 && x<4)(y>1 && y<3) T1 (x>2 && x<4)T2 (y>1 && y<3) W1_clicked() tap(1, 5) tap(3, 5) tap(3, 2)

Example: Music Player App ❖ ❖ ❖ ❖ ❖ ❖ ❖ ❖ ❖ ❖ ❖

Two subproblems Generate individual events Generate sequences of events

Generating Sequences of Events Concatenate individual events generated by concolic execution.

Baseline Algorithm S, Set of all event sequences s.t. each sequence takes a unique path Baseline algorithm Set of covered branches Goal: cover these

Baseline Algorithm Suffers from Path Explosion Number of sequences generated for Music Player app by baseline algorithm

ACTEve Algorithm ACTEve: Automated ConcolicTesting of Event-driven programs

ACTEve Algorithm S, Set of all event sequences s.t. each sequence takes a unique path ACTEve algorithm Baseline algorithm R s.t. R ⊆ S Set of covered branches Goal: cover these ACTEve is relatively sound

Path Subsumption Program state in concolic execution Maps memory location to values (symbolic or concrete) Path constraint

Path Subsumption Note - memory map – path constraint Program entry Path • Path subsumes

Path Subsumption Note - memory map – path constraint Program entry Path • Path subsumes - Don’t generate test corresponding to any path that is an extension of - Only generate tests corresponding to paths that are extension of

Path Subsumption • Checking path subsumption is very expensive in general • Constraint implication check • Matching memory map • But, path subsumption can be checked cheaply in special cases • Read-only events • Events whose mutual ordering does not matter • etc.

Read-only Events Program Entry eventis does not write to any memory location. corresponds to is subsumed by q corresponds to Path executed for event sequence

Read-only Events ❖ ❖ ❖ ❖ ❖ ❖ ❖ ❖ ❖ ❖ ❖ Read-only events are represented as ❖

ACTEve System Architecture

Empirical Study • Apply ACTEve and baseline algorithms • event sequences of length up to 4 • 16 concurrently running emulators • time budget of 12 hours • Measured three metrics • running time • number of feasible paths • number of satisfiability checks

Empirical Results

Future Work Widget Explosion

Main Contributions • Concolic execution to generate individual events • ACTEve: an efficient algorithm for bounded exhaustive testing of event-driven programs • Requires only a small fraction (5-36%) of time compared to baseline algorithm • Implementation for Android

Backup slides

Read-only Events Program Entry becausedoesnot write to any memory location. corresponds to event sequence corresponds to in Path executed for input event sequence

A Solution: Use Platform-specific Knowlege Output of Android’s “Hierarchy Viewer” tool

A Solution: Use Platform-specific Knowlege Output of Android’s “Hierarchy Viewer” tool • void onTouchEvent(MotionEvent e) { int rawX = (int) e.getX(); int rawY = (int) e.getY(); int x = (rawX – MARGIN) / SIZE; int y = (rawY – MARGIN) / SIZE; if (x >= 0 && x < 3 && y >= 0 & y < 3) { int cell = x + 3 * y; …}

Path Subsumption Program Entry Program Entry same program location • Path Path {is feasible} {is feasible} Covered branches Covered branches

Path Subsumption Program Entry Program Entry same program location • Path Path if we explore all paths that extends , then no need to explore any path that extends because no additional branch coverage will be obtained. {is feasible} {is feasible} Covered branches Covered branches

Example: Music Player App Path constraint when PAUSE button is tapped on

Automated Concolic Testing of Smartphone Apps