1 / 41

introduction to CTS

introduction to CTS. Sep 2, 2010 Jingtao. agenda. back ground CTS history CTS work flow download, install and run CTS check CTS report typical fail cases. background.

raven
Download Presentation

introduction to CTS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. introduction to CTS Sep 2, 2010 Jingtao

  2. agenda • back ground • CTS history • CTS work flow • download, install and run CTS • check CTS report • typical fail cases

  3. background • Google define an “Android compatibility” device as one that can run any application written by third-party developers using the Android SDK and NDK. • Device that are properly compatible can seek approval to use Android trademark and Android Market client software. 5

  4. how to verify compatibility • the Android Compatibility Definition Document(CDD) spells out the specific device configuration that will be considered compatible. • CTS(an automatically compatibility test tool) test • Third party application test 6

  5. the process to get certification • Google release CTS according to each Android platform publicly. • manufacturer run CTS on their device. • manufacture fix any bug found by CTS. • if manufacture think any CTS case is wrong, he should upload the CTS report and detail description to Google's database. • if Google accept manufacture's description, he will grant wavier to that device. • Google will think the device is Android compatible if it pass all the CTS case or all the fail cases have waiver. 7

  6. requirement in CDD • software API/UI compatibility • reference app test • application packaging compatibility • multimedia compatibility • developer tool compatibility • hardware compatibility • performance compatibility • security model compatibility • system update

  7. hardware compatibility • display • keyboard • Non-touch navigation • screen orientation • touch-screen input: must have a touch screen • USB: must support adb and mass storage mode • navigation keys: function of home, menu and back must be available to user at any time. • wireless data networking: must support for at least one wireless data standard capable of 200kbit/sec or greater. • camera: must include a rear-facing camera, at least 2 megapixels • accelerometer: must include a 3-axis accelerometer and must be able to deliver events at 50 Hz or greater. • compass: must include a 3-axis compass and must be able to deliver events 10 Hz or greater. • GPS: must include a GPS receiver • telephony • memory and storage: /data partition must be at least 150MB and must have at least 92MB memory. • application shared storage: must be at least 2GB in size. • Bluetooth: must include a Bluetooth transceiver and enable the RFCOMM-based Bluetooth API.

  8. What is CTS • Compatibility Test System • open source • commercial-grade(20000+ cases)‏ • Automation compatibility test tool

  9. what does it test • Signature • Platform API • Dalvik VM • Platform Data Model • Intents • Permissions • Resources

  10. CTS History • Android1.0 - 16784 cases • Cupcake - 21383 • Donut - 22756 • Eclair r3 - 23138 • Froyo r2 - 23830

  11. CTS work flow

  12. download CTS • http://source.android.com/compatibility/downloads.html 1, git ls-remote --heads git://192.168.1.250/android-mirror/platform/manifest.git ... refs/heads/android-cts-2.1_r3 refs/heads/android-cts-2.1_r4 refs/heads/android-cts-2.2_r1 refs/heads/android-cts-2.2_r2 ... 2, repo init -u git://192.168.1.250/android-mirror/platform/manifest.git -b android-cts-2.2_r2 3, repo sync cts

  13. build CTS • svn://192.168.1.250/oms/branches/horse make BUILD_CTS=true cts

  14. install CTS • install Android SDK • unzip android-cts-2.1_r3-x86.zip android-cts |-- docs |-- repository `-- tools |-- cts.jar `-- startcts • edit android-cts/tools/startcts, set • SDK_ROOT=<absolute fold of Android SDK>

  15. run CTS connect phone to PC android-cts/tools/startcts to enter CTS shell 3 mode to start test: • plan mode • package mode • case mode

  16. Sample of CTS commands • start --plan CTS • start --plan Android -p android.app.cts.ActivityManagerTest • start --plan Android -t android.app.cts.ActivityManagerTest#testGetRunningServices

  17. what does CTS do install test case apk (adb -s deviceID install -r ...apk)‏ execute test (am instrument -w -e [para] <component>...)‏ uninstall test case apk (adb -s deviceID uninstall ...)‏

  18. run CTS under Windows 1. unzip CTS package and android SDK package to your windows PC.2. Modified the SDK_ROOT in cts\android-cts\tools\startcts :SDK_ROOT=D:/CTS/sdk/android-sdk_r05-windows/android-sdk-windows/tools3. use below command to enter CTS shell: java -Xmx512M -cp D:\CTS\cts\android-cts\tools\cts.jar; D:\CTS\cts\android-cts\tools\hosttestlib.jar;D:\\lib\ddCTS\cts\android-cts\tools\junit.jar;D:\CTS\sdk\android-sdk_r05-windows\android-sdk-windows\tools\lib\ddmlib.jar com.android.cts.TestHost D:\CTS\cts\android-cts\repository\host_config.xml Please change red marked to you current file path.known issue:The file (cts_result.css, cts_result.xsl, logo.gif, newrule-green.png ) under CTS result can not automatically copy in new session folder.Attention:If you need to clear the data in addon emulator, Please use: emulator @addon -wipe-data

  19. config phone before run CTS empty SD card • never Screen Timeout • English locale • install and set input method to LatinIME • open WIFI and GPRS • correct date/time • /opl/etc/mountd.conf, system.prop • CtsDelegatingAccessibilityService.apk

  20. test report • firefox repository/results/20xx.../testResult.xml

  21. Fail Cases

  22. Case Study: CTS on FBW1.4 • Waivered • Internal resource ID: 38 cases • Locale related: 11 cases • Others: 4 cases • Fixed certification time: 3 cases • Fixed camera parameter: 1 case • UI related( For example, check box layout): 6 cases • Mock Locations: 22 cases (just test in eng build)‏ • Waiting for waiver • Google Search dependant: 2 cases • no popup window when application force close

  23. Public Resource ID • http://developer.android.com/reference/android/R.html • Frameworks/base/core/res/res/values/public.xml public static final int accelerate_decelerate_interpolator Constant Value: 17432580 (0x010a0004)‏

  24. Internal Resource ID out/target/common/R/com/android/internal/R.java com.android.internal.R.id.alertTitle cupcake: 0x0102014e borqs: 0x01020167 168         String expected = dateFormat.format(mCalendar.getTime()); 169         TextView tv = (TextView) d.getWindow().findViewById(com.android.internal.R.id.alertTitle); 170         assertEquals(expected, tv.getText());

  25. resource ID • R.id.contentPanel • com.android.internal.R.id.topPanel • R.id.alertTitle • R.id.custom

  26. Mock Location • 84 // test that mock locations are allowed so a more descriptive error message can be logged • 85 if (Settings.Secure.getInt(mContext.getContentResolver(), • 86 Settings.Secure.ALLOW_MOCK_LOCATION, 0) == 0) { • 87 fail("Mock locations are currently disabled in Settings - this test requires " • 88 + "mock locations"); • http://androidappdocs.appspot.com/guide/topics/location/index.html • during initial development in the emulator, you may not have access to real data from a real location provider (Network or GPS). In that case, it may be necessary to spoof some data for your application using a mock location provider.

  27. Settings-> Applications-> Development

  28. check box

  29. camera • android.hardware.cts.CameraTest#testAccessParameters junit.framework.AssertionFailedError: expected:<2048> but was:<2560> at android.hardware.cts.CameraTest.assertParameters(CameraTest.java:392)‏ 381 private void assertParameters(Parameters parameters) { ... 387 final int ORIGINALPICWIDTH = 2048; 388 final int ORIGINALPICHEIGHT = 1536; 389 390 // Before Set Parameters 391 assertEquals(PixelFormat.JPEG, parameters.getPictureFormat()); 392 assertEquals(ORIGINALPICWIDTH, parameters.getPictureSize().width);

  30. Case Study: CTS on OPhone 2.0 Case number: 23107 cases Nexus One Oscar-r42369 S1-r49137 Fail cases: 3 34 5 Execut time: 6 h 6 h 4 h

  31. fail case summary on Oscar • internal resource ID: 17 • UI: 8. (check box, grid layout, list padding, pixel buffer)‏ • phone number utils: 4 • network type: 1 • SD card write permission: 1 • CTS case issue: 3

  32. phone number util cases(92585)‏ • android phone do not have sys.min.match.digits but can pass • set sys.min.match.digits to 7 can pass the case but phone will not match incoming call correctly. • e.g. create user A(tel: 01150539) and user B(mobile:13701150539) in sequence in Contacts, when mobile 13701150539 incoming call, phone will always show user A incorrectly on screen.

  33. Sdcard write permission case • android.permission.cts.NoSdCardWritePermissionTest • android.permission.WRITE_EXTERNAL_STORAGE uid=1000: sysetm user gid=1015: Sdcard write group SDcard: android oms(before) oms(fixed)‏ 075 777 775

  34. Ipv6 related fail • tests.api.java.net.SocketTest#test_getLocalAddress junit.framework.AssertionFailedError: ANY address not returned correctly (getLocalAddress) with preferIPv6Addresses=true, preferIPv4Stack=false 0.0.0.0/0.0.0.0:49380 at tests.api.java.net.SocketTest.test_getLocalAddress(SocketTest.java:645)‏

  35. known failure actually passed • android.provider.cts.ContactsTest#testGroupMembershipTable bug 2258907, needs investigation will pass after install gms. the gms package link: smb://192.168.0.146/share/gms user: b184 pass: zhanghui

  36. rule to pass CTS • Do not modify framework. • review with SDK team before modify • run related CTS cases after modify

  37. summary • CTS can block SA • manufacture will submit ticket relate to CTS • we can help you setup CTS env

  38. 41

More Related