1 / 44

Development of a Web Mapping Application Using Open Source Tools and Cloud Computing Services

Development of a Web Mapping Application Using Open Source Tools and Cloud Computing Services. Matt Bertrand. Project Goal. Compare expected turnout to actual turnout per voting ward on election day Prioritize last-minute get-out-the vote efforts.

javier
Download Presentation

Development of a Web Mapping Application Using Open Source Tools and Cloud Computing Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development of a Web Mapping Application Using Open Source Tools and Cloud Computing Services Matt Bertrand

  2. Project Goal • Compare expected turnout to actual turnout per voting ward on election day • Prioritize last-minute get-out-the vote efforts. • Assess reliability of expected turnout estimates. • Analyze effectiveness of campaign’s get-out-the vote effort.

  3. Requirements • Enter, edit, and view expected and actual votes cast per ward • Display data on a map and table simultaneously • Import and export vote data • Click a ward on the map to bring up its data for editing/viewing • Click a ward in the table to zoom in & highlight that ward on map • Refresh map as soon as data is edited

  4. Requirements • Choose between displaying expected turnout, actual turnout and expected:actual turnout ratio on the map • Automatically calculate ratio from entered data • Security: require login to the site. Two separate login accounts - one for editing, one for viewing only • Support ??? users (scalable).

  5. Components • Virtual Server Machines • Amazon Web Services • Open Source GIS tools • PostgreSQL & PostGIS • OpenLayers • GeoServer • Other Open Source Tools • Apache web server • Tomcat Java application server • PHP DataGrid

  6. Amazon Web Services • Elastic Compute Cloud (EC2) • Provides resizable computing capacity • Designed to make web-scale computing easy & accessible to developers • Create and use “Amazon Machine Images” (AMI) • Virtual server machines • Many templates available • Choose your operating system (Linux, Windows) • Choose your horsepower (RAM, processor, etc).

  7. Amazon Web Services • Elastic Block Store • Persistent storage for Amazon EC2 instances (AMI’s). • Ideal for databases • Easily switch an EBS from one AMI to another • Use to swap database from a small AMI to a faster, more powerful AMI

  8. PostgreSQL and PostGIS • PostgreSQL • Relational database system • PostGIS • Spatial objects for PostgreSQL • Store, query, and manipulate spatial data objects

  9. GeoServer • Open source geospatial server • Written in Java and the J2EE platform • Includes a graphical user interface for configuration • Web Mapping Service • Web Feature Service

  10. Web Map & Feature Services • Web Mapping Service (WMS) • Request: specifies the geographic layer(s) and area of interest to be processed. • Response: returns geo-registered map images (JPEG, PNG, etc), and/or layer attributes. • Web Feature Service (WFS) • Request: specifies the geographic layer(s) and feature subset of interest based on spatial and/or non-spatial queries • Response: return description and/or attributes of the specified layer features. • Also capable of updating, creating, and deleting features

  11. OpenLayers • JavaScript library for displaying map data • Provides an API (Application Programming Interface) for building web-based geographic applications. • Makes use of map “tiles” • Pre-rendered small sections of map, improves performance • Can load data from many sources: • Google Maps, Yahoo Maps, Virtual Earth • GeoServer, MapServer • Any Web Map Service or Web Feature Service

  12. PHP DataGrid • Free data-bound grid control written in PHP • View, edit, and export data • Customizable – select which database fields to display, turn editing on/off for each field, etc.

  13. Putting it all together AWS GeoServer ? OpenLayers = PostGIS Apache/ Tomcat PHPDataGrid

  14. Architectural Overview Amazon Web Services OpenLayers Tomcat

  15. AWS Scalability X-Large AMI X-Large AMI Small AMI EBS EBS Data PostGreSQL Data PostGreSQL GeoServer GeoServer Apache Apache Small AMI: 1.7 GB RAM 1 32-bit Virtual Core = 1 EC2 Compute Unit Extra Large AMI: 15 GB RAM 4 64-bit Virtual Cores = 8 EC2 Compute Units Light Load Setup Potential Heavy Load Setup

  16. AWS Setup • Register for an account • Not free - cost is based on use of AMI’s and storage • ~ $2/day for a small running AMI instance • Pick a management console • ElasticFox addon for Firefox • Amazon now provides its own online console • Manage AMI and EBS instances

  17. AWS Setup • Select and start an AMI • Operating System: Ubuntu Linux • Size: small instance (1.7 GB RAM, 160 GB drive, 1 virtual CPU equivalent to 32-bit 1.2 GHz Intel Xeon) • Log in to the AMI using SSH (secure shell) • Install and configure required software • Tomcat, Apache, Apache-Tomcat Connector, GeoServer, PostgreSQL, PostGIS, PHP

  18. AWS Setup • Create an Elastic Block Store (EBS) • 1 GB – plenty enough to hold NH ward data • Maximum size: 1 TB • Attach the EBS to the running AMI instance • Mount the EBS as a volume • Create a PostGIS database on the mounted volume

  19. Database Setup • A GIS shapefile of ward boundaries was provided by the NH campaign staff • The shapefile was imported into PostgreSQL using a built-in PostGIS command (shp2pgsql) • Added new fields (turnout_expected, turnout_actual, turnout_ratio) • Created indexes • Created a trigger to automatically calculate the ratio whenever expected or actual turnout values changed

  20. Database Setup • PostgreSQL UPDATE trigger CREATE OR REPLACE FUNCTION updateratio() RETURNS trigger AS IF NEW.turnout_expected > 0 THEN NEW.turnout_ratio = NEW.turnout_actual / NEW.turnout_expected; ELSE NEW.turnout_ratio = 0; END IF; RETURN NEW;

  21. GeoServer Configuration • Create a new DataStore • Stores connection information on PostgreSQL database • Create a new FeatureType • Stores information on a geographic vector feature stored in the database (projection, boundaries, fields, etc). • Create Styled Layer Descriptors • XML format for specifying how to draw features on a map

  22. SLD Rule Example <sld:Rule> <sld:Name>1.6 - 1.8</sld:Name> <sld:Title>1.6 - 1.8</sld:Title> <ogc:Filter> <ogc:PropertyIsBetween> <ogc:PropertyName>turnout_ratio</ogc:PropertyName> <ogc:LowerBoundary> <ogc:Literal>1.600001</ogc:Literal> </ogc:LowerBoundary> <ogc:UpperBoundary> <ogc:Literal>1.8</ogc:Literal> </ogc:UpperBoundary> </ogc:PropertyIsBetween> </ogc:Filter> <sld:PolygonSymbolizer> <sld:Fill> <sld:CssParameter name="fill">#FF6F00</sld:CssParameter> <sld:CssParameter name="fill-opacity">1</sld:CssParameter> </sld:Fill> <sld:Stroke> <sld:CssParameter name="stroke">#6E6E6E</sld:CssParameter> <sld:CssParameter name="stroke-width">0.4</sld:CssParameter> <sld:CssParameter name="stroke-opacity">1</sld:CssParameter> </sld:Stroke> </sld:PolygonSymbolizer> </sld:Rule>

  23. Web Page Layout • Requirement: Display map and table/grid simultaneously Main page (index.html) Map <iframe> PHP DataGrid page (gridedit.php)

  24. OpenLayers • Requirement: Display ward data on a map var wmsnh = new OpenLayers.Layer.WMS("NH Wards 4326", “http://[....]-amazon.aws.com/geoserver/wms", { layers: 'topp:nh_wards4326', styles: sld, srs: 'EPSG:4326', format: 'image/png', tiled: true, transparent: true }, { 'opacity': 0.75, 'isBaseLayer': false });

  25. OpenLayers • Requirement: Click a ward on the map to bring up its data to edit/view map.events.register('click', map, function(e) { var url = "/geoserver/wfs?request=GetFeature&version=1.0.0&typeName=topp:nh_wards4326&outputFormat=json&FILTER=%3CFilter%20xmlns=%22http://www.opengis.net/ogc%22%20xmlns:gml=%22http://www.opengis.net/gml%22%3E%3CIntersects%3E%3CPropertyName%3Ethe_geom%3C/PropertyName%3E%3Cgml:Point%20srsName=%22EPSG:4326%22%3E%3Cgml:coordinates%3E" + lonlatGCS.lon + "," + lonlatGCS.lat + "%3C/gml:coordinates%3E%3C/gml:Point%3E%3C/Intersects%3E%3C/Filter%3E"; OpenLayers.loadURL(url, '', this, getGridPage); OpenLayers.Event.stop(e); });

  26. OpenLayers • WFS Request <Filter xmlns="http://www.opengis.net/ogc" xmlns:gml="http://www.opengis.net/gml"> <Intersects> <PropertyName>the_geom</PropertyName> <gml:Point srsName="EPSG:4326"> <gml:coordinates>-75.12, 40.25 </gml:coordinates> </gml:Point> </Intersects> </Filter>

  27. OpenLayers • JSON (JavaScript Object Notation) • GeoJSON: JSON for geographic data { "type": "FeatureCollection", "features": [ { "type": "Feature", "id": "nh_wards4326.48", "properties": { "objectid": 48, "fips": 7160, "name": "Pittsburg", "twn_ward": null, "ncec_code": "3300761780", "turnout_expected": 3000, "turnout_actual": 1000, "turnout_ratio": 0.33, }, "geometry": { "type": "MultiPolygon", "coordinates": ………… } ] }

  28. OpenLayers • Requirement: Highlight selected feature on map • First, create a highlight style and empty vector layer var highlight_style = { strokeColor: 'Red', strokeWidth: 4, strokeOpacity: 1, fillOpacity: 0.0 }; //Add highlight vector layer for selected features hilites = new OpenLayers.Layer.Vector("Highlighted", { isBaseLayer: false, features: [], visibility: true, style: highlight_style });

  29. OpenLayers • Add the selected feature to the ‘Highlighted’ vector layer function highlightFeature(req) { var features = new OpenLayers.Format.GeoJSON(out_options).read(req.responseText); // have the Vector layer purge its feature list, replace them with the new one hilites.destroyFeatures(); hilites.addFeatures(features); hilites.setVisibility(true); }

  30. OpenLayers • Requirement:Click a ward in the table to zoom in & highlight on map function highlightFeature(req) { … if (zoomOnSelect) { bounds = features[0].geometry.getBounds(); map.zoomToExtent(bounds); }

  31. OpenLayers • Requirement: switch between maps of actual turnout, expected turnout, and turnout ratio function changeSLD(sldChosen) { saveBounds(); sld = sldChosen; init(); document.getElementById("legendImg").src = '/geoserver/wms?REQUEST=GetLegendGraphic&VERSION= 1.0.0&FORMAT=image/png&WIDTH=100&HEIGHT=20&LAYER= topp:nh_wards4326&sld=/geoserver/styles/' + sld + '.sld'; }

  32. Data Import • Requirement: Import voter turnout data from an external file into the database. • Comma-delimited text file • NCEC Code, expected turnout, actual turnout

  33. Data Import • PHP import code snippet $handle = fopen($uploadfile, "r"); while (($data = fgetcsv($handle)) !== FALSE) { $result = pg_prepare($connection, "update" . $data[0], 'UPDATE nh_wards SET turnout_expected = $2, turnout_actual=$3 WHERE ncec_code = $1'); $result = pg_execute($connection, "update" . $data[0], array($data[0],$data[1],$data[2])); } } fclose($handle);

  34. Security • Requirement: Login with password to enter site • Two login roles – edit and view • Logins and passwords stored in ‘users’ table in database • PHP code used to validate user input against database values • User forwarded to either data edit page or view page depending on login role.

  35. Final Step • Bundle the AMI • Saves current AMI setup as a new template • New AMI’s based on this template will be identical with all software installed and configured. • Avoid having to reinstall PostgreSQL, Geoserver, etc. all over again whenever a new instance is created.

  36. Development Cost & Effort • Cost: $110.41 • Amazon EC2 instance: $110.15 • Amazon EBS storage: $0.26 • Time: 4 weeks • ~ 100 hours total • Energy:

  37. Resources • Amazon Web Services – aws.amazon.com • GeoServer – geoserver.org • OpenLayers – openlayers.org • PostGIS - postgis.refractions.net • PostgreSQL – postgresql.org • Demo – openwebmap.com/votemap

More Related