Build Your Own Data GEO Viewer in the Cloud

1. Build Your Own Data.gov GEO Viewer in the Cloud Brand Niemann http://semanticommunity.net January 20, 2011

2. Overview • 1. Background • 2. Data.gov GEO Viewer • 3. USGS OpenView Map Viewer • 4. Google Earth • 5. US EPA National Geospatial Program • 6. Geospatial OneStop • 7. Spotfire Silver • 8. Geospatial Platform • 9. Sample Applications • 9.1 US EPA National Geospatial Program • 9.2 Geospatial One Stop • 9.3 Places and Spaces Atlas of Science • 9.4 NGA-NCOIC • 9.5 Build Your Own Data.gov GEO Viewer in the Cloud • 10. Some Next Steps

3. 1. Background • At the Web Site: http://data.gov • There is the Catalog: Raw Data, Tools, and Geodata. • My Suggestion: What About All Three Integrated?! • There are Multiple File Types: XML, CVS/Text, KML/KMZ, Shapefile, Other? • My Suggestion: Why Not Try Using Spotfire for all of these!

4. 1. Background • At the Web Site: http://datagov.ideascale.com/ • There is the Suggestion: http://datagov.ideascale.com/a/pmd/27611-6440 • developer wiki by michael.daconta • His Suggestion: I think it is important to have a separate wiki set up for developers as there are many developer topics where collaboration is key. • At the Wiki Site: http://semanticommunity.info/ • There is the Work: http://semanticommunity.info/Data.gov/GEOViewer • Build Your Own Data.gov GEO Viewer in the Cloud by myself (and actually many others that I have collaborated with as you will see) • My Suggestion: Why not try doing this with the new MindTouch Technical Documentation Suite (Wiki) and TIBCO Spotfire Silver, both cloud computing tools!

5. 1. Background • 2011 Geospatial Committee, Emerging Technology SIG, ACT/IAC: • Geospatial Layer of any FEA Implementation: • Where are we today? • “How do we bring potential users — those we don’t think of as conventional users of geospatial data such as program or acquisition officers — and help them make better decisions, using tools that help visualize data geographically.” (Karen Siderelis, DoI GIO, Government Computer News, March 5, 2010). • Where do standards come in? • Metadata, Web services, and other standards are important. • Ontology and its role. • See http://cegis.usgs.gov/ontology.html • Semantic interoperability and its potential. • See http://www.ogcnetwork.net/geosemantics • Outcome – operating model? • “Groups have their own infrastructures and technologies. We really need cloud computing, it provides a unique opportunity to leverage our technology investment to build a shared infrastructure.” (Jerry Johnson, EPA GIO, Government Computer News, March 5, 2010). • Next steps. • A geospatial platform in the cloud that includes the above!

6. 1. Background • 2011 Geospatial Committee, Emerging Technology SIG, ACT/IAC: • Geospatial Layer of any FEA Implementation: • How best to engage with the FEA Geospatial Profile? • My suggestion: Add the Geospatial Cloud Platform to the FEA Geospatial Profile. • Where can emerging geospatial technologies and practices benefit the FEA? • My suggestion: By supporting the Geospatial Cloud Platform. • Develop a mission statement for the subcommittee. • My suggestion: Applying emerging geospatial technologies to the Geospatial Cloud Platform. • Produce a graphic of geospatial in enterprise architecture. • Use Geospatial Platform Conceptual Model and Geospatial Platform Conceptual Architecture and Role of FEA (See slides 25 and 26). • Develop workshop for the March meeting. • My suggestion: Invite Jerry Johnson, EPA GIO, to provide an update on the Geospatial Platform & then have a discussion on how the ETSIG can support that. • Look at semantics in the FEA and Geospatial Layer. • http://www.ogcnetwork.net/geosemantics • Look for opportunities for collaboration with SOCoP. • http://semanticommunity.info/Spatial_Ontology_Community_of_Practice

7. 2. Data.gov GEO Viewer http://www.data.gov/whatsnew/geoviewer

8. 2. Data.gov GEO Viewer • We are pleased to announce the availability of the Data.gov GEO Viewer, an interactive mapping tool designed to let users preview geospatial data available through the Data.gov catalogs. Using this tool, you can immediately view datasets on an interactive map, overlay them with other datasets, and investigate the underlying data. While not a full featured Geographic Information System, this new capability allows users to quickly determine whether datasets in the Data.gov catalogs are suitable for their needs before taking the time to download them for further exploration and use. Note: This was not “immediately” at least for the first two. • To get started, simply click on the "Preview" link where it appears on a dataset's metadata page. Please note that not all geospatial data in the Data.gov catalog is available for preview at this time. The Preview link is included for those datasets that are currently supported. Note: See next slide. • We hope that you find the Data.gov GEO Viewer to be a useful tool in helping you search for, use, and understand the geospatial data holdings of Data.gov. Please give us feedback on the Viewer, and let us know what new features and tools you would like to see in the future. Note: Do not find section for Data.gov GEO Viewer! Comment: I made the suggestion about a year ago that Data.gov try Spotfire for viewing and working with geospatial data and now I am going to show what I have done with it.

9. 2. Data.gov GEO Viewer EPA Region 1 Environmentally Sensitive Areas

10. 3. USGS OpenView Map Viewer Note: I Put the Map Layer Metadata in a Spreadsheet and in Spotfire. http://mapdss1.er.usgs.gov/openview/

11. 4. Google Earth Request Access: http://csms.terradex.com/PublicPages/Application.aspx

12. 5. US EPA National Geospatial Program Note: I used these in Section 9. http://www.epa.gov/enviro/geo_data.html

13. 5. US EPA National Geospatial Program • Keyhole Markup Language (KML) is an XML schema for expressing geographic annotation and visualization within Internet-based, two-dimensional maps and three-dimensional Earth browsers. KML was developed for use with Google Earth, which was originally named Keyhole Earth Viewer. It was created by Keyhole, Inc, which was acquired by Google in 2004. The name "Keyhole" is an homage to the KH reconnaissance satellites, the original eye-in-the-sky military reconnaissance system first launched in 1976. KML is an international standard of the Open Geospatial Consortium. Google Earth was the first program able to view and graphically edit KML files. • The KML file specifies a set of features (place marks, images, polygons, 3D models, textual descriptions, etc.) for display in Google Earth, Maps and Mobile, or any other 3D Earth browser (geobrowser) implementing the KML encoding. Each place always has a longitude and a latitude. • KML files are very often distributed in KMZ files, which are zipped files with a .kmz extension. • Note: Need to convert KML/KMZ files to CSV for use in Spotfire. • David Smith @ EPA recommends: http://choonchernlim.com/kmlcsv/ Source: http://en.wikipedia.org/wiki/Keyhole_Markup_Language

14. 6. Geospatial OneStop Note: I put this in Spotfire (325 MB). See slide 16. http://gos2.geodata.gov/wps/portal/gos

15. 6. Geospatial OneStop • Geospatial One-Stop makes it easier, faster, and less expensive for all levels of government and the public to access geospatial information. The geodata.gov portal is designed to facilitate communication and sharing of geographic data and resources to enhance government efficiency and improve citizen services. • Statistics (I put these in Spotfire) • Publish • Community • Marketplace • Kinds of Data • Personalization • Benefits • The portal is a catalog of geospatial information containing thousands of metadata records (information about the data) and links to live maps, features, and catalog services, downloadable data sets, images, clearinghouses, map files, and more. The metadata records were submitted to the portal by government agencies, individuals, and companies, or by harvesting the data from geospatial clearinghouses. Sources: http://gos2.geodata.gov/gos/static/default/faq.htm and Help Center.

16. 6. Geospatial OneStop Web Player

17. 7. Spotfire Silver See Terms of Use. Limited to Three Spreadsheets of Less Than 10 MB. http://goto.spotfire.com/g/?KXLMSEWRM9

18. 7. Spotfire Silver • Demo Files: • An Example to Get Started! • Map - Europe Departments • Map - Latin American States • Map - Middle Eastern States • Map - SE Asia Provinces • Map - US States and Counties (see next slide) • Map – World Counties and Cities (see slide 21) • Private Analysis: Two Applications Allowed

19. 7. Spotfire Silver PC Desktop

20. 7. Spotfire Silver Web Player

21. 7. Spotfire Silver Note: This combines mapping and statistical analyses and visualizations! http://semanticommunity.info/Sustainable_Society_Foundation_Index_2008

22. 7. Spotfire Silver • Example of Spotfire Community Help that I needed for the previous application where the country names in the Sustainable Society Foundation Index did not exactly match the country names in the World County Boundary File database. • How to combine two data sets from different sources via a matching table? • We have two tables with information about companies we would like to combine. Unfortunately the company name column does not match 100%. We have to create a matching table to be able to combine both datasets. It is easy in Spotfire to merge these 3 tables into one. You can follow these simple steps and re-apply it to your own data. • First load both data tables into Spotfire (e.g. TOP100-companies-Germany and DAX-companies) • Create a Matching Table out of the fields "Firmenname" and "Company" and load this into Spotfire as well. • Add a new table from Existing and duplicate the TOP100 table, call it TOP100-companies-Germany and DAX • Insert Column from External Data and chose the Matching Table, add the "Company" name in a full outer join • Again Insert Column from External Data and choose the DAX-companies table. • You have now added the information if the TOP100 company is a DAX company or not. Keep the final table linked to its source tables. You can also link the source data tables to their data sources, but embed the matching table. When you do this, the resulting table will be refreshed whenever the Spotfire files is opened or refreshed/reloaded. • Source: http://spotfire.tibco.com/community/forums/t/1384.aspx • Spotfire File in Web Player: http://ondemand.spotfire.com/public/ViewAnalysis.aspx?file=/Users/TIBCEVL-52522/Public/DAX-and-TOP100

23. 8. Geospatial Platform Note: I put the Key Documents in MindTouch (see next slide). http://www.geoplatform.gov/index.html

24. 8. Geospatial Platform http://semanticommunity.info/Data.gov/GEOViewer/Geospatial_Platform_Key_Documents

25. 8. Geospatial Platform Source: http://semanticommunity.info/Data.gov/GEOViewer/Geospatial_Platform_Key_Documents#Figure_1._Geospatial_Platform_Conceptual_Model

26. 8. Geospatial Platform Source: http://semanticommunity.info/Data.gov/GEOViewer/Geospatial_Platform_Key_Documents#Figure_C.1._Geospatial_Platform_conceptual_architecture_and_role_of_FEA

27. 8. Geospatial Platform Note: I put this in a table in Spotfire. http://www.geoplatform.gov/gulfresponse/

28. 8. Geospatial Platform Mashup of GeoPlatform for Gulf Response and EPA Monitoring Data Web Player

29. 8. Geospatial Platform Submitted Comment: Use tools like Spotfire that support both geospatial and statistical visualizations and Open Data where you can easily get the data into and back out. http://geoplatform.ideascale.com/

30. 9. Sample Applications • 9.1 US EPA National Geospatial Program • 9.2 Geospatial One Stop • 9.3 Places and Spaces Atlas of Science • 9.4 National Geospatial-Intelligence Agency (NGA) -Network Centric Operations Industry Consortium (NCOIC) • 9.5 Build Your Own Data.gov GEO Viewer in the Cloud

31. 9.1 US EPA National Geospatial Program The "map" looks okay, except for the Guam Brownfields site way to the east. The Longitude should be negative? http://semanticommunity.info/Data.gov/GEOViewer

32. 9.2 Geospatial One Stop Statistics from Web Site) PC Desktop (Web Player)

33. 9.3 Places and Spaces Atlas of Science http://semanticommunity.info/Atlas_of_Science

34. 9.4 NGA-NCOIC Source: http://semanticommunity.info/@api/deki/files/1200/=NGA_to_NCOIC_Brief_Final_Slides_21Jun10dlb.pdf

35. 9.4 NGA-NCOIC • Tuesday, March 1, 2011: • 0800-1000 Focused Session - NGA/NCOIC Cloud Computing Proposal. • Description: NCOIC has been asked to submit a formal proposal to the National Geospatial-Intelligence Agency (NGA) to design, build and present a demonstration on how cloud computing technologies and/or techniques could be used to enhance interoperability between geospatial information systems. This session will review the draft proposal for NGA -- all are encouraged to attend and participate. Source: https://www.ncoic.org/home/

36. 9.5 Build Your Own Data.gov GEO Viewer in the Cloud USGS Openview Map Viewer Map Layer Metadata as of Novmber 17, 2010. PC Desktop (Web Player)

37. 9.5 Build Your Own Data.gov GEO Viewer in the Cloud http://semanticommunity.info/Data.gov/GEOViewer

38. 10. Some Next Steps • Work with the USGS Spatial Ontology (RDF/OWL): • http://cegis.usgs.gov/ontology.html • SICoP Monthly Webinars – To Be Announced Soon. • January 27, 2011: ET SIG Meeting. • January 30, 2011: Atlas of Science Final Submission due for March 25-26, 2011, VIVO Workshop. • March 1, 2011: Focused Session on NGA/NCOIC Cloud Computing Proposal. • March 16-17, 2011: European Spotfire Conference. • April 12, 2011: 11th SOA for E-Gov Conference. • June 5-9, 2011: 2011 SemTech Conference.