1 / 17

Our Data, Ourselves Hack-Day

Our Data, Ourselves Hack-Day. Giles Greenway Tobias Blanke Jenifer Pybus Mark Cote. Department of Digital Humanities. The Project:. What and how much data do smartphone apps collect? What can it say about us and how is it used? What do we think about this? Can we put it to better use?

ebonyj
Download Presentation

Our Data, Ourselves Hack-Day

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Our Data, Ourselves Hack-Day Giles Greenway Tobias Blanke Jenifer Pybus Mark Cote Department of Digital Humanities

  2. The Project: • What and how much data do smartphone apps collect? • What can it say about us and how is it used? • What do we think about this? • Can we put it to better use? • ~20 Young Rewired State coders issued with Android 'phones. • Custom MobileMiner app reports on app usage. • Sends data to a modified CKAN instance. • CKAN is written in Python, based on the Pylons framework. • Released by the Open Knowledge Foundation: http://ckan.org/

  3. The Data: MobileMiner App: http://kingsbsd.github.io/MobileMiner . • Poll /proc/<pid>/net/<tcp/udp> • Look for sockets/ports. • Count transmitted/received bytes. • GSM cell ids. • Mobile and wireless networks. • App notifications. • Periodically save data to an internal SQLite database that users can access. • Upload data to a CKAN instance .

  4. GSM Cell Tower Locations: http://opencellid.org • Full GPS is too invasive, and consumes power. • Avoid use of Google location API. • OpenCellId provides locations of (many) cell towers. • Currently include UK database within the app. • Next: Bridge MobileMiner to cell DB via CKAN API?

  5. CKAN:

  6. CKAN:

  7. Processing The Data: • Aggregate app usage per user per day. • Cluster GSM cells by k-means using SciKitLearn Python library • Label clusters using OpenStreetMaps. • Gather app data by scraping the Play Store. (BeautifulSoup, PhantomJS & Selenium )

  8. Docker: https://www.docker.com/ • Docker Linux Containers: Dockerfile->Image->Container • Installs CKAN, packages, libraries. • Link to containers for Postgress and Solr. • Create users and database tables. • Provide access to the data via Ipython Notebooks. • Provide tools like Numpy, SciKitLearn and NLTK. • Allows users to experiment. • Documents the software environment. • Allows for easy deployment. • Free public image hosting.

  9. Questions: • Can we link app usage to physical locations? • Can we make use of cells whose locations are unknown? • Can we cluster on a spatial AND temporal basis? • Do apps with certain permissions use more data?

  10. The Line!

  11. Getting an .apk package: http://apps.evozi.com/apk-downloader/

  12. Fighting Back? • Grab the app's .apk package file from a rooted phone? • Decompress the package and examine AndroidManifest.xml. • Decompile the app and examine the source code.

  13. Fighting back: Decompressing the .apk: http://code.google.com/p/android-apktool/ apktool d com.onetouchgame.TheLine.apk

  14. AndroidManifest.xml <receiver android:enabled="true" android:name="com.simplecreator.app.RemoteNotificationReceiver"> <intent-filter> <action android:name="cn.jpush.android.intent.REGISTRATION"/> <action android:name="cn.jpush.android.intent.UNREGISTRATION"/> <action android:name="cn.jpush.android.intent.MESSAGE_RECEIVED"/> <action android:name="cn.jpush.android.intent.NOTIFICATION_RECEIVED"/> <action android:name="cn.jpush.android.intent.NOTIFICATION_OPENED"/> <action android:name="cn.jpush.android.intent.ACTION_RICHPUSH_CALLBACK"/> <category android:name="com.onetouchgame.TheLine"/> </intent-filter> </receiver> <service android:name="com.umeng.update.net.DownloadingService" android:process=":DownloadingService"/> <activity android:name="com.umeng.update.UpdateDialogActivity" android:theme="@android:style/Theme.Translucent.NoTitleBar"/> • The app receives intents from the push notification service jpush.cn. There is a mobile analytics service. • Is that why it had open sockets on port 3000? .

  15. Fighting Back: Decompile the App http://code.google.com/p/dex2jar/ dex2jar.sh com.onetouchgame.TheLine Decompile the .jar file:

  16. Fighting Back: “The Usual Suspects” Look for PhoneStateListeners and LocationListeners: if (paramLocation != null) { d1 = paramLocation.getLatitude(); d2 = paramLocation.getLongitude(); boolean bool1 = d1 < 29.999998211860657D; Classes provided by tencent.com (a mobile ad service) reference latitutude and longitude. Classes provided by jpush.cn and umeng.com also reference LocationListeners.

  17. Download our app: Follow us on Twitter: @KingsBSD Read our blog: Slideshare: http://www.slideshare.net/kingsBSD/

More Related