Serving Geographic Data

1. Serving Geographic Data A Paul R Cooper MAGIC

2. Introduction Paul Cooper

3. House-Keeping • One day course • 4 sessions • 1000 – 1100 • Coffee • 1115 – 1230 • Lunch • 1330 – 1500 • Tea/Coffee • 1530 – 1700

4. Introduction Overview of System Standards Data Stores Introduction to Tomcat Geoserver Client side software Exercise Close Course Outline

5. What’s in it for You! • Allows you to put data on the web. • Standards compliant techniques. • Makes your data accessible to a large audience.

6. Handouts, notes and questions • Handouts: • Copies of PowerPoint slides • Space for notes • Exercise plans • Please ask questions at the end of sections • Text-books and manuals available

7. System Overview

8. Overview of Web services • User interfaces with Client Software. • Client Software talks to Server software. • Server software contained by Web server. • Server Software gets data from Data Store. Client Web Server Geoserver Datastore

9. Standards

10. Why should we be bothered? • Everyone in my field uses <insert data format> • International standards allow cross-discipline data sharing. • NERC is an interdisciplinary organization. • Environmental scientists need to share data across disciplinary boundaries • Discipline specific formats do not promote data sharing across disciplines

11. Standards • All web services depend on standards • Geographic data use OGC standards • (Open Geospatial Consortium) • ISO TC211 standards also important • OGC and ISO are convergent • Both depend on W3C standards • XML, CSS, HTML, XHTML, DOM

12. OGC standards: • Web Feature Service • http://www.opengeospatial.org/standards/wfs • Web Map Service • http://www.opengeospatial.org/standards/wms • Styled Layer Descriptor • http://www.opengeospatial.org/standards/sld • GML (also ISO 19136) • http://www.opengeospatial.org/standards/gml • Filter Encoding • http://www.opengeospatial.org/standards/filter

13. ISO Standards • Plenty to choose from! • Full list at http://www.isotc211.org/ • Ones that are of interest to us are: • ISO 19110 (Feature Cataloguing). • ISO 19111 (Spatial Referencing by Coordinates) • ISO 19112 (Spatial Referencing by Geographic identifier) • ISO 19115 and ISO 19139 (Metadata) • Plus lots more. • ISO standards not free, unfortunately.

14. Web standards • XML • http://www.w3.org/XML/ • HTML and XHTML • http://www.w3.org/MarkUp/ • CSS • http://www.w3.org/Style/CSS/ • Document Object Model (DOM) • http://www.w3.org/DOM/

15. Other standards • ECMAScript • Javascript implements ECMAScript. • Depends on DOM. • Unfortunately widely extended. • http://www.ecma-international.org/publications/standards/Ecma-262.htm • Java, JavaServer Pages, etc. • SQL (ISO 9075 with OGC extensions) • KML (Google proprietary standard)

16. Demystifying XML

17. A brief guide to XML <myTag>this is some data</myTag> • XML is a mark up language • Always starts: • <?xml version="1.0" encoding="UTF-8"?> • Uses tags to define elements. • Tags may have attributes. • Must have opening and closing tags or have “/” before closing “>”. • Attribute values must be in quotes. • Tags must be nested correctly. • Must have single “Root” tag. • More! <myTagmyAttribute= “Paul” /> <myTag> <myName> Paul </myName> </myTag>

18. Datastores

19. Datastores • Any means of storing geographic information • Spatially enabled RDBMS • PostGIS/PostgreSQL • MySQL • Oracle (Oracle Spatial not required) • ArcSDE (We do have this, but proprietary) • File based • Shapefiles • Images • GML • Service based – other WFS servers

20. PostGreSQL • This course will use PostgreSQl/PostGIS • Respected OpenSource database • Stable and reliable fully featured SQL RDBMS • http://www.postgresql.org/ • http://postgis.refractions.net/

21. PostgreSQL utilities • psql – SQL command line • pgAdmin III • GUI interface for managing database. • Shp2pgsql • Utility to import shapefiles into PostGIS • Gshp2pgsql • GUI interface for shp2pgsql • psqlODBC • ODBC driver – used for connecting to Access etc.

22. Creating PostGIS Data Source • Standard SQL database • Need one or more columns of “geometry” type • Geometry columns mustbe registered in “geometry_columns” table. • Simplest way is to create the table then use the “addgeometrycolumn” function. • May be necessary to insert entry manually into “geometry_columns” table.

23. Example CREATE TABLE penguins ( penguin_id INTEGER, penguin_name VARCHAR, penguin_type VARCHAR, latitude DOUBLE, longitude DOUBLE); SELECT AddGeometryColumn('penguins', 'penguin_point', 3031, 'POINT', 2 );

24. Geometry Types • POINT • LINESTRING • POLYGON • MULTIPOINT • MULTILINESTRING • MULTIPOLYGON • GEOMETRYCOLLECTION

25. Spatial reference ID (SRID) • This is a reference to a “Well Known” set of identifiers for projections and other spatial reference systems. • The only one in widespread use is the EPSG database (http://www/epsg.org) • An SRID should be prefaced with a namespace identifier (e.g. EPSG::3031), but often isn’t!

26. Useful SRIDs • 4326: Geographic coordinates on WGS84 datum • 7405: British National Grid • 3031: Standard Antarctic Polar Stereographic projection • 3204-3293: “North Up” projections for each IMW map tile in Antarctica

27. Geometry Constructors • Geometry columns are binary • You can’t insert data directly into them • Use geometry constructor functions • E.G. INSERT INTO myTable (my_geom) values (GeomFromText(‘POINT(-50 -60)',4326));

28. Exercises 1 & 2 Import spatial data into PostGreSQL Psql –h localhostWebCoursenogcn

29. Maintaining Geometry

30. Maintaining a Geometry column • Geometry columns are binary and not editable. • You need to input and edit latitude and longitude • Geometry is updated with UPDATE and INSERT trigger functions. • Trigger functions can make use of the geometry functions of PostGIS

31. PGPLSQL to convert lat/long to point • NEW.the_geom := ST_GeomFromText('POINT('||NEW.londec||' '||NEW.latdec||')',4326); • OR, if you want to use a different SRID: • NEW.the_geom := ST_Transform( ST_GeomFromText('POINT('||NEW.londec||' '||NEW.latdec||')',4326),3031);

32. SQL to build lines and polygons • Select ST_MakeLine(the_geom) as the_line from (select the_geom from MyPoints where lineid=1234 order by timestamp) pnt; • Several polygon creation functions: • ST_MakePolygon(linestring, [linestring[]]) • ST_BuildArea(geometry) • ST_Polygonize(geometry set)

33. More on geometry_columns table • Created by the system to store metadata about geometry fields. • Used by other software to determine the properties of geometry fields. • Geoserver won’t detect geometry fields unless this is correctly maintained.

34. Using Views with PostGIS • Create View using usual SQL commands • Can include geometry manipulation functions • MANUALLY add entry to “geometry_columns” table

35. Example • CREATE VIEW penguin_tracks as Select pnt.penguin_name, ST_MakeLine(penguin_point) as penguin_line from (select penguin_name, penguin_point from penguins order by penguin_id) pnt group by pnt.penguin_name; • INSERT into geometry_columns values (‘’,’public’,’penguin_tracks’,’penguin_line’,2,3031,’LINESTRING’);

36. Exercise 3 & 4 Maintaining Geometry Columns Creating lines from points

37. Introduction to TOMCAT

38. Introduction to TOMCAT • Tomcat is a web server • Provided by Apache Foundation • Free/Open-Source software • Can work alongside Apache web server • Provides additional technologies • Java Servlet container • Java Server Pages

39. TOMCAT file structure

40. Tomcat Management • Application Management • Start, Stop • Reload • Restarts an application • Reloads Java classes • Undeploy • Expire session • Timeout for inactive session • Load new applications

41. Geoserver

42. Geoserver • Geoserver is a Java application that runs in a Java servlet container. • We run it in Tomcat; it can run on other servers that provide support for Java servlets. • Provides OGC compliant • Web Map Service (WMS 1.1.1) • Web Feature Service (WFS 1.0) • Web Coverage Service (WCS 1.0) • The reference implementation for WFS 1.0 • More information at www.geoserver.org

43. Geoserver File Structure

44. Status! Geoserver Management • Done through Web pages • Important bits! Configuration (Password protected) Demonstration

45. Steps to create a feature source • Namespace • Data Source • Style • Feature Types

46. Namespace • A namespace makes your feature names unique • “Coast” is unlikely to be unique • “MyNameSpace::Coast” will be unique • Namespace definitions have two parts • A name (e.g. add) • A URL, which need not exist but should be in a domain you control (e.g. www.add.scar.org) • A standard XML technique.

47. Namespace creation tool First Second

48. Data Source • Basic capability: • PostGIS • Shapefiles • Web Feature Server • Optional download • Indexed Shapefile • DB2 • Oracle • ArcSDE • MySQL (β) • VPF (β) • MapInfo (β)

49. Setting up PostGISDataSource • You need to supply: • A name for this data source • A namespace to use with this data source • The PostGreSQL host machine address • Use Localhost on the teaching PCs • postgres.nerc-bas.ac.uk is the BAS server • The name of the database • A userid and password for the database

50. Set up screen for PostGres Database