Entertainment Platform

1. EE575 Lecture8 Smart Phone Programming Entertainment Platform 2011. 9. 29 Sensors/Location/Camera

2. Sensors

3. What we are going to implement is … Show all sensors available on the device Accelerometer output Magnetometer output SensorMonitor

4. What we are going to learn is … • Sensor class • SensorManager system service • How to get a system service • How to get all available sensors • How to get a specific sensor • E.g., accelerometer or magnetometer • SensorEvent class • How to get sensor output • SensorEventListener interface • How to register and unregister a sensor event listener

5. Sensorclass • Class representing a sensor • Android defines various types of sensors • This lecture covers an accelerometer and a magnetic field sensor

6. Definition of the Coordinate • The X axis is horizontal and points to the right, the Y axis is vertical and points up and the Z axis points towards the outside of the front face of the screen Gravity

7. SensorManager class • Android provides a system service, SensorManager, which lets you access the device's sensors • SensorManagersensorManager = (SensorManager)getSystemService(SENSOR_SERVICE);

8. SensorEvent class • This class represents a Sensor event and holds information such as the sensor's data, type, time-stamp, and accuracy Sensor values (next slide)

9. SensorEventclass (cont’) • The float[] values field of SensorEvent class contains sensor output values. • The length and contents of the values array depends on which sensor type is being monitored • values for an accelerometer • values[0]: acceleration on the x-axis (m/s2) • values[1]: acceleration on the y-axis (m/s2) • values[2]: acceleration on the z-axis (m/s2) • values for magnetometer • values[0]: ambient magnetic field in the x-axis (uT) • values[1]: ambient magnetic field in the x-axis (uT) • values[2]: ambient magnetic field in the x-axis (uT) uT: micro-Tesla. Unit of magnetic field, which is known as “magnetic flux density”

10. SensorEventListener Interface • Used for receiving notifications from the SensorManager when sensor values have changed • Make your activity receive notifications • publicclassSensorMonitorActivityextends Activity implementsSensorEventListener{ … } • Two methods must be overrided to implement SensorEventListener interface

11. Layout: Three TextViews • <?xml version="1.0" encoding="utf-8"?> • <LinearLayout • xmlns:android="http://schemas.android.com/apk/res/android" • android:orientation="vertical" • android:layout_width="fill_parent" • android:layout_height="fill_parent" • > • <TextView • android:id="@+id/txt_available_sensors" • android:layout_width="fill_parent" • android:layout_height="wrap_content" • android:text="Available Sensors:\n" • /> • <TextView • android:id="@+id/txt_acc_out" • android:layout_width="fill_parent" • android:layout_height="wrap_content" • android:text="Accelerometer Output:\n" • /> • <TextView • android:id="@+id/txt_mag_out" • android:layout_width="fill_parent" • android:layout_height="wrap_content" • android:text="Magnetometer Output:\n" • /> • </LinearLayout>

12. Permission NO permission setup is required for sensors 

13. Main Activity Class • A class must implement SensorEventListener interface to receive notifications on sensor data change • onSensorChanged() and onAccuracyChanged() must be overrided SensorMonitorActivity.java • publicclassSensorMonitorActivityextends Activity implementsSensorEventListener{ • ... • /** Called when the accuracy of a sensor has changed **/ • @Override • publicvoidonSensorChanged(SensorEvent event) { ... } • /** Called when sensor values have changed **/ • @Override • publicvoidonAccuracyChanged(Sensor sensor, int accuracy) { ... } • ... • }

14. Variables SensorMonitorActivity.java • publicclassSensorMonitorActivityextends Activity implementsSensorEventListener { • /** Sensor objects **/ • private Sensor accelerometer; • private Sensor magnetometer; • /** SensorManager system service **/ • privateSensorManagersensorManager; • /** A list containing all sensors available on the device **/ • private List<Sensor> sensorList; • /** TextViews for output **/ • privateTextViewtxtAvailableSensors; • privateTextViewtxtAccOut; • privateTextViewtxtMagOut; • /** Data buffer holding sensor output **/ • privatefloat[] accValues; • privatefloat[] magValues; • /** Moving average coefficients **/ • privatefloatalphaAcc = 0.2f; • privatefloatalphaMag = 0.3f; • ...

15. onCreate() SensorMonitorActivity.java • publicvoidonCreate(Bundle savedInstanceState) { • super.onCreate(savedInstanceState); • setContentView(R.layout.main); • /** Get views **/ • txtAvailableSensors= (TextView)findViewById(R.id.txt_available_sensors); • txtAccOut= (TextView)findViewById(R.id.txt_acc_out); • txtMagOut= (TextView)findViewById(R.id.txt_mag_out); • /** Get SensorManager system service **/ • sensorManager= (SensorManager)getSystemService(SENSOR_SERVICE); • /** Get all sensors installed in the device **/ • sensorList= sensorManager.getSensorList(Sensor.TYPE_ALL); • /** Initialize data buffer **/ • accValues= newfloat[3]; • magValues= newfloat[3]; • /** Print out information of all available sensors **/ • for(ListIterator<Sensor> li = sensorList.listIterator(); li.hasNext(); ) { • Sensor s = (Sensor)li.next(); • txtAvailableSensors.append(s.getName() + "\n"); • } • }

16. onCreate() SensorMonitorActivity.java • ... • @Override • publicvoidonCreate(Bundle savedInstanceState) { • super.onCreate(savedInstanceState); • setContentView(R.layout.main); • /** Get views **/ • txtAvailableSensors= (TextView)findViewById(R.id.txt_available_sensors); • txtAccOut= (TextView)findViewById(R.id.txt_acc_out); • txtMagOut= (TextView)findViewById(R.id.txt_mag_out); • /** Get SensorManager system service **/ • sensorManager= (SensorManager)getSystemService(SENSOR_SERVICE); • /** Get all sensors installed in the device **/ • sensorList= sensorManager.getSensorList(Sensor.TYPE_ALL); • /** Print out information of all available sensors **/ • for(ListIterator<Sensor> li = sensorList.listIterator(); li.hasNext(); ) { • Sensor s = (Sensor)li.next(); • txtAvailableSensors.append(s.getName() + "\n"); • } • } • @Override • publicvoidonResume() { • ... private List<Sensor> sensorList; HTC Desire

17. Warning!! • Always make sure to disable sensors you don't need, especially when your activity is paused. • Failing to do so can drain the battery in just a few hours. • The android system will NOTdisable sensors automatically when the screen turns off. @Override publicvoidonResume() { /** Register SensorEventListener: Turn on sensing **/ } @Override publicvoidonPause() { /** Unregister SensorEventListener: Turn off sensing **/ }

18. onResume() andonPause() SensorMonitorActivity.java • publicvoidonResume() { • super.onResume(); • /** Get sensors **/ • accelerometer= sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER); • magnetometer= sensorManager.getDefaultSensor(Sensor.TYPE_MAGNETIC_FIELD); • /** Register SensorEventListener: Turn on sensing **/ • sensorManager.registerListener(this, accelerometer, SensorManager.SENSOR_DELAY_GAME); • sensorManager.registerListener(this, magnetometer, SensorManager.SENSOR_DELAY_GAME); • } • publicvoidonPause() { • super.onPause(); • /** Unregister SensorEventListener: Turn off sensing **/ • sensorManager.unregisterListener(this); • } What is this? SensorEventListener? Why? SensorManager

19. Sensing Rate (Delay) sensorManager.registerListener(this, accelerometer, SensorManager.SENSOR_DELAY_GAME); sensorManager.registerListener(this, magnetometer, SensorManager.SENSOR_DELAY_GAME); • The actual sampling rate depends on the sensor type and device • Sensor.TYPE_ACCELEROMETER • SENSOR_DELAY_NORMAL 161ms • SENSOR_DELAY_UI 20ms • SENSOR_DELAY_GAME 20ms • SENSOR_DELAY_FASTEST 20ms • Sensor.TYPE_MAGNETIC_FIELD • SENSOR_DELAY_NORMAL 125ms • SENSOR_DELAY_UI 60ms • SENSOR_DELAY_GAME 20ms • SENSOR_DELAY_FASTEST 16.7ms Measured in Nexus S

20. onAccuracyChanged() • The main activity must implements SensorEventListener interface • publicclassSensorMonitorActivityextends Activity implementsSensorEventListener { … } SensorMonitorActivity.java • /** Called when the accuracy of a sensor has changed **/ • @Override • publicvoidonAccuracyChanged(Sensor sensor, int accuracy) { • /** Nothing to do with this method **/ • }

21. onSensorChanged() SensorMonitorActivity.java • /** Called when sensor values have changed **/ • @Override • publicvoidonSensorChanged(SensorEvent event) { • /** Output format **/ • String format = "####.###"; • DecimalFormatdf = newDecimalFormat(format); • StringBuildersb; • switch(event.sensor.getType()) { • caseSensor.TYPE_ACCELEROMETER: • /** EMA: Exponential Moving Average **/ • accValues[0] = alphaAcc*accValues[0] + (1 - alphaAcc)*event.values[0]; • accValues[1] = alphaAcc*accValues[1] + (1 - alphaAcc)*event.values[1]; • accValues[2] = alphaAcc*accValues[2] + (1 - alphaAcc)*event.values[2]; • sb = newStringBuilder(); • sb.append("Accelerometer Output:\n"); • sb.append("X-axis: " + df.format(accValues[0]) + "\n"); • sb.append("Y-axis: " + df.format(accValues[1]) + "\n"); • sb.append("Z-axis: " + df.format(accValues[2]) + "\n"); • txtAccOut.setText(sb); • break; • caseSensor.TYPE_MAGNETIC_FIELD: • ...

22. onSensorChanged() (cont’) SensorMonitorActivity.java • publicvoidonSensorChanged(SensorEvent event) { • ... • switch (event.sensor.getType()) { • caseSensor.TYPE_ACCELEROMETER: • ... • break; • caseSensor.TYPE_MAGNETIC_FIELD: • sb= newStringBuilder(); • /** EMA: Exponential Moving Average **/ • magValues[0] = alphaMag*magValues[0] + (1 - alphaMag)*event.values[0]; • magValues[1] = alphaMag*magValues[1] + (1 - alphaMag)*event.values[1]; • magValues[2] = alphaMag*magValues[2] + (1 - alphaMag)*event.values[2]; • sb.append("Magnetometer Output:\n"); • sb.append("X-axis: " + df.format(magValues[0]) + "\n"); • sb.append("Y-axis: " + df.format(magValues[1]) + "\n"); • sb.append("Z-axis: " + df.format(magValues[2]) + "\n"); • txtMagOut.setText(sb); • break; • } • }

23. Sensor Summary • Make your activity implement SensorEventListener to receive notifications • Use SensorManager system service to • Get available sensors • Register and unregister event listener callback functions • Always make sure to disable sensors you don't need, especially when your activity is paused. • Register on onCreate(), unregister on onPause() (or onStop()) • Override SensorEventListener.onSensorChanged() which is automatically called when sensor values have changed

24. Location

25. What we are going to implement is … GPS control MapView Image markers on the visited locations Toast popup showing new location

26. What we are going to learn is … • MapView, MapControllerclasses • How to use Google map APIs • LocationManager system service • How to get a system service • How to get the best location provider • LocationListenerinterface • How to get location information • GeoPoint, Location classes • How to represent a location

27. MapView & MapController • MapView is a View which displays a map with data obtained from the Google Maps service. • MapViewcaptures key presses and touch gestures to pan and zoom the map. It can also be controlled programmatically via MapController. • A MapView can only be constructed (or inflated) by a MapActivity. • Your main activity must extend MapActivity. • MapViewmapView = (MapView)findViewById(R.id.mapview); • MapControllermapController= mapView.getController(); • mapController.setZoom(16); • mapController.setSatellite(false); • mapController.setBuiltInZoomControls(true); • mapController.animateTo(newGeoPoint(0, 0));

28. Location &GeoPoint classes • GeoPoint • Represents a pair of latitude and longitude, stored as integer numbers of microdegrees. • Google map APIs represent a location using a GeoPoint • Location • Represents a pair of latitude and longitude, stored as float numbers of degrees, as well as other information, such as sensing time • Used by LocationManager and other android systems Locationlocation; doublelatitude = location.getLatitude(); double longitude = location.getLongitude(); GeoPointnewPoint = newGeoPoint((int)(latitude*1e6), (int)(longitude*1e6));

29. LocationManager class • Provides access to the system location services • LocationManagerlocationManager= (LocationManager)getSystemService(Context.LOCATION_SERVICE); • Gives the name of location providers • StringbestLocProvider= locationManager.getBestProvider(criteria, true); • Allows applications to obtain periodic updates of the device's geographical location • locationManager.requestLocationUpdates(bestLocProvider, 10000L, 1.0f, this);

30. requestLocationUpdates() public void requestLocationUpdates(String provider, long minTime, float minDistance, LocationListenerlistener)

31. Criteriaclass • A class indicating the application criteria for selecting a location provider. Example of selecting a location provider /** Get LocationManager system service **/ LocationManagerlocationManager = (LocationManager)getSystemService(Context.LOCATION_SERVICE); /** Initialize criteria for choosing the location provider **/ Criteria criteria= new Criteria(); criteria.setAccuracy(Criteria.NO_REQUIREMENT); criteria.setPowerRequirement(Criteria.NO_REQUIREMENT); /** Get the best location provider that meets the criteria **/ StringbestLocProvider= locationManager.getBestProvider(criteria, true); /** Register location update event listener **/ locationManager.requestLocationUpdates(bestLocProvider, 10000L, 1.0f, this);

32. Criteriaclass • A class indicating the application criteria for selecting a location provider. Example of selecting a location provider /** Get LocationManager system service **/ LocationManagerlocationManager = (LocationManager)getSystemService(Context.LOCATION_SERVICE); /** Initialize criteria for choosing the location provider **/ Criteria criteria= new Criteria(); criteria.setAccuracy(Criteria.NO_REQUIREMENT); criteria.setPowerRequirement(Criteria.NO_REQUIREMENT); /** Get the best location provider that meets the criteria **/ StringbestLocProvider= locationManager.getBestProvider(criteria, true); /** Register location update event listener **/ locationManager.requestLocationUpdates(bestLocProvider, 10000L, 1.0f, this);

33. LocationListener Interface • Used for receiving notifications from the LocationManager when the location has changed. • These methods are called if the LocationListener has been registered with the location manager service using the requestLocationUpdates(String, long, float, LocationListener)

34. Google API Key (1/3) C:\Users\test\.android>"%java_home%\bin\keytool.exe" -list -keystoredebug.keystore -keypas s android -storepass android -alias androiddebugkey androiddebugkey, 2010. 2. 6, PrivateKeyEntry, 인증서 지문(MD5): AA:22:33:F7:32:12:22:EC:68:93:8B:55:44:69:ED:47

35. Google API Key (2/3) http://code.google.com/intl/ko-KR/android/add-ons/google-apis/maps-api-signup.html

36. Google API Key (3/3)

37. Layout main.xml • <?xml version="1.0" encoding="utf-8"?> • <com.google.android.maps.MapView • xmlns:android="http://schemas.android.com/apk/res/android" • android:id="@+id/mapview" • android:layout_width="fill_parent" • android:layout_height="fill_parent" • android:clickable="true" • android:apiKey=“ABCDEFGHIJKLMNOPabcdefghigklmnop1234567" • /> API key received from Google

38. Permission Required to use Google map APIs AndroidManifest.xml • ... • <uses-library android:name="com.google.android.maps" /> • </application> • <uses-permission android:name="android.permission.INTERNET" /> • <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" /> • <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" /> • </manifest> • ACCESS_COARSE_LOCATION • Allows an application to access coarse (Cell-ID, WiFi) location • ACCESS_FINE_LOCATION • Allows an application to access fine (e.g., GPS) location

39. Main ActivityClass • Extend MapActivity to use Google map APIs • isRouteDisplayed() must be overrided • Implement LocationListener interface to receive notifications on location change • onLocationChanged(), onProviderDisabled(), onProviderEnabled(), onStatusChanged() must be overided GoogleMapTestActivity.java • publicclassGoogleMapTestActivityextendsMapActivityimplementsLocationListener { • ... • protectedbooleanisRouteDisplayed() { ... } • publicvoidonLocationChanged(Location location) { ... } • publicvoidonProviderDisabled(String provider) { ... } • publicvoidonProviderEnabled(String provider) { ... } • publicvoidonStatusChanged(String provider, int status, Bundle extras) { ... } • ... • }

40. Variables GoogleMapTestActivity.java • publicclassGoogleMapTestActivityextendsMapActivityimplementsLocationListener { • /** A View which displays a Google map **/ • privateMapViewmapView; • /** A utility class to manage panning and zooming of a map. **/ • privateMapControllermapController; • /** The default zoom level **/ • privatestaticfinalintdefaultZoomLevel = 16; • /** Location service manager **/ • privateLocationManagerlocationManager; • /** Criteria for selecting a location provider **/ • private Criteria criteria; • /** The name of the best location provider **/ • private String bestLocProvider; • /** An ImageView showing a marker on the map **/ • privateImageViewmapMarker; • /** A Layout that describes the marker's layout **/ • privateMapView.LayoutParamslayoutParams; • ...

41. onCreate() GoogleMapTestActivity.java • publicvoidonCreate(Bundle savedInstanceState) { • super.onCreate(savedInstanceState); • setContentView(R.layout.main); • mapView= (MapView)findViewById(R.id.mapview); • mapView.setBuiltInZoomControls(true); • mapView.setSatellite(false); • /** Get map controller for zooming in or out the map view **/ • mapController= mapView.getController(); • /** Set the default zoom level **/ • mapController.setZoom(defaultZoomLevel); • locationManager= (LocationManager)getSystemService(Context.LOCATION_SERVICE); • /** Initialize criteria for choosing the location provider **/ • criteria= new Criteria(); • criteria.setAccuracy(Criteria.NO_REQUIREMENT); • criteria.setPowerRequirement(Criteria.NO_REQUIREMENT); • /** Get the best location provider **/ • bestLocProvider= locationManager.getBestProvider(criteria, true); • /** Register location update event listener **/ • locationManager.requestLocationUpdates(bestLocProvider, 10000L, 1.0f, this); • }

42. onLocationChanged() GoogleMapTestActivity.java • publicvoidonLocationChanged(Location location) { • double latitude = location.getLatitude(); • double longitude = location.getLongitude(); • GeoPointnewPoint = newGeoPoint((int)(latitude*1e6), (int)(longitude*1e6)); • mapController.animateTo(newPoint); • mapController.setZoom(defaultZoomLevel); • layoutParams = newMapView.LayoutParams( • LayoutParams.WRAP_CONTENT, • LayoutParams.WRAP_CONTENT, • newPoint, • MapView.LayoutParams.CENTER • ); • mapMarker = newImageView(this); • mapMarker.setImageResource(R.drawable.tree); • mapView.addView(mapMarker, layoutParams); • Toast.makeText(this, latitude + ", " + longitude, Toast.LENGTH_SHORT).show(); • }

43. Other methods GoogleMapTestActivity.java • @Override • protectedbooleanisRouteDisplayed() { • /** Nothing to do with this method **/ • returnfalse; • } • @Override • publicvoidonProviderDisabled(String provider) { • /** Nothing to do with this method **/ • } • @Override • publicvoidonProviderEnabled(String provider) { • /** Nothing to do with this method **/ • } • @Override • publicvoidonStatusChanged(String provider, int status, Bundle extras) { • /** Nothing to do with this method **/ • }

44. Location Summary • UseMapView, MapControllerclasses to use Google map APIs • LocationManagersystem service • How to get a system service • How to get the best location provider • ImplementLocationListener interface to receive location-change events • onLocationChagned()

45. Camera

46. App. to implement

47. Basic: Camera Class • Android supports Camera class for camera application development • Set image capture settings • Start/stop preview • Snap pictures • Retrieve frames for encoding video.

48. Basic: Permission Setup • To access the device camera, you must declare the CAMERA permission in your Android Manifest. • Include the <uses-feature> manifest element to declare camera features used by your application. AndroidManifest.xml If you want to use the auto-focus feature

49. Basic: Camera Methods • Camera class supports useful static methods • Camera.open() • Obtain an instance of Camera • Camera.takePicture() • Capture a photo • Camera.startPreview() • Start updating the preview surface. (What is the surface?) • Preview must be started before you can take a picture • Camera.stopPreview() • Stop updating the preview surface. • Camera.release() • Release the camera for use by other applications.

50. Basic: SurfaceView • To use the preview method of Camera class, you need to use SurfaceView. • SurfaceView • Provides a dedicated drawing surface embedded inside of a view hierarchy. • Provides better drawing performance than the ordinary View class, and thus, is suitable for camera previewing. • In this example, create a new class MyPreview, which extends SurfaceView to use the preview method of Camera class