340 likes | 481 Views
ICEDS demo – preparing an OpenGIS Web Map Server. Oliver Greening, ESYS plc Jeremy Morley, Prof. Jan-Peter Muller Dept. Geomatic Engineering, UCL Presented by Rob Scott ESYS plc. Start-up Aims. To prepare an OpenGIS Web Map Server + website To demonstrate Web access to large datasets
E N D
ICEDS demo – preparing an OpenGIS Web Map Server Oliver Greening, ESYS plc Jeremy Morley, Prof. Jan-Peter MullerDept. Geomatic Engineering, UCL Presented by Rob Scott ESYS plc
Start-up Aims • To prepare an OpenGIS Web Map Server + website • To demonstrate Web access to large datasets • Europe (to Urals) and Africa • To provide feedback on data preparation process • Eventual aim: to use open access (I.e. open source / freeware / free to download) software • Deegree: deegree.sourceforge.net • Current implementation: vendor software
Available Datasets • 3 arcsec. unedited SRTM DEM data • Downloadable from Eros Data Center (EDS) • VMAP-0 vector data • Downloadable from NGA (was NIMA) • NASA MODIS "Blue Marble" global composite • Downloadable from NASA • (Elsewhere too in easier format but not instant ftp) • JPL Landsat 5 full-res mosaic of Africa • 15 DVD-ROMs! • Courtesy of Nevin Bryant
Datasets URLs • 3 arcsec. unedited SRTM DEM data • ftp://edcftp.cr.usgs.gov/pub/data/srtm/Eurasia/ • ftp://edcftp.cr.usgs.gov/pub/data/srtm/Africa/ • VMAP-0 vector data • http://geoengine.nima.mil • NASA MODIS "Blue Marble" global composite • ftp://mitch.gsfc.nasa.gov/pub/stockli/bluemarble/
Areas of Interest • Two areas selected for demonstration: • British Isles (UK and Ireland) • Madagascar • AOIs: • Madagascar:43º East - 51 º East; 11 º South - 26 º SouthExpanded area: 30ºE-55ºE; 5ºS-30 • British Isles:11º West - 2º East; 49º North - 61º NorthExpanded area: 11ºW - 15ºE; 45ºN - 65ºN
SRTM processing • Download ZIPped DEM tiles (1º x 1º) • Madagascar: 85 tiles; British Isles: 182 tiles • Unzip tiles • Import into ArcInfo grids (adapted AML from EDC, invoked from a Unix shell script) • Merge grids into 1º latitudinal strips • Merge strips into a full DEM grid of AOI • Create a mask grid from DCW polygonised coastline • 1 inside coastline; NODATA outside
SRTM processing (cont.) • Mask DEM to set to NODATA all areas outside coastline • Project DEM to British National Grid • ArcInfo requires projected system for hill shading • choice of projection not too critical • Hill-shade the DEM • Selected a factor of 5 exaggeration for British Isles; factor of 3 for Madagascar • Project hill-shaded grid back to Geographic • NB: Important to use nearest-neighbour resampling
Blue Marble processing • Import files into ArcInfo (or into ENVI) • Available from EDC as 2 binary data files – eastern and western hemisphere • 30arcsec resolution: 21600 x 21600 pixels • Although text files stated 'interleaved', did not specify interleaving type • In fact (third attempt!), files are BIP (Band Interleaved by Pixel) • For ArcInfo, requires a HDR header file to be constructed (straightforward – simple text format) • Merge eastern and western hemispheres
Web Map Server • Using Cadcorp SIS 6.0 in Win2K3 Server running Microsoft IIS 6 for this demo • Why? • Very simple to set up OGC WMS: • Open datasets and symbolise in desktop application • Save a "Saved Window Definition" (SWD) project file • Include ISAPI interface DLL + SIS DLLs in directory • Set up as a Virtual Directory in IIS • The maps are automatically served as they appear in the desktop system
Data formats • Cadcorp 6 will read a large variety of data formats • However not done anything clever in subdividing the datasets => LARGE! • => memory and speed of access issues • => some data compression needed • Choice: for this demo, use compressed data formats
Data formats (2) • Hill-shaded DEMs: PackBits compressed TIFF • British Isles: 62.8 MB • Madagascar: 72.4 MB • Blue Marble global composite: PackBits TIFF • Extracts taken of extended AOIs: • British Isles/NW Europe: 22.6MB • Madagascar/E Africa: 39.2MB • VMAP-0: original VPF data (no import) • Cadcorp will read this vector format • NW Europe: 532MB (though mostly unused layers) • Sothern Africa: 362MB (also mostly unused layers)
On-line Services • OpenGIS services can be made to 'cascade' • The map server receives layers for its map from other OpenGIS services • E.g.:JPL OnEarth server: • http://onearth.jpl.nasa.gov • Landsat 7 global mosaic • No data holding or data preparation • However, relies on secondary server being up and responsive, and on fast network link
JPL Landsat 5 Africa mosaic • 15 DVD-ROMs • 13 DVD-ROMs holding 30m mosaic • Other two holding, e.g. subsampled versions • Data stored as 5º x 5º tiles • 1 file per band; 3 bands (Landsat 1,2,3) • 324MB per file • GeoTIFF format (also, easy to construct HDR files) • Madagascar: 18 files (i.e. 6 tiles x 3 bands)
Landsat 5 mosaic import • Copy from DVD and unzip • Create HDR files (ArcInfo doesn't read GeoTIFF) • NB: ArcInfo Workstation on Solaris • Import each band per tile to ArcInfo grid • Create an image stack for each tile • Quick – logical operation only • Merge tiles • Export to image format (JPEG) • 65.8MB for Madagascar (40-50ºE; 15-30ºS) • 6 – 20MB per tile, unmerged
Current total data holding • ~ 35.9 GB • Including intermediate products • Very little archiving or compression of intermediate products yet!
Web site • WMS images are generated in response to a URL containing map specification, e.g.: • http://webtest.ge.ucl.ac.uk/iceds/SisISAPI.dll?REQUEST=GETMAP&VERSION=1.1.0&SRS=EPSG:4326&STYLES=&WIDTH=400&HEIGHT=400&FORMAT=IMAGE/JPEG&EXCEPTIONS=IMAGE/JPEG&LAYERS=COMPLETE---0,COMPLETE---1,COMPLETE---4,&BBOX=-10.9074225697008,45.1003094517055,6.48085238195591,62.4885844033622 • Requires a web site to provide user interface controls for map browsing, layer control, etc.
Web site (2) • Current web site served from an average PC • Client-side processing only HTML 3.2 (basic!) • Forms for UI controls • Tables for interface layout • Server-side processing by a simple ASP script • ASP = Active Server Pages • Server-side scripting in Microsoft IIS web server • VBScript language embedded in HTML page • IIS server processes VBScript on demand • VBScript outputs HTML in place in document
Web site (3) • Current interface would require only minor changes to work with another WMS package • WMS interface is only through URL • URL is standardised by OpenGIS specifications • Only root of URL to specify active component and its virtual directory change
Web site (4) • http://webtest.ge.ucl.ac.uk/iceds/SisISAPI.dll?REQUEST=GETMAP&VERSION=1.1.0&SRS=EPSG:4326&STYLES=&WIDTH=400&HEIGHT=400&FORMAT=IMAGE/JPEG&EXCEPTIONS=IMAGE/JPEG&LAYERS=COMPLETE---0,COMPLETE---1,COMPLETE---4,&BBOX=-10.9074225697008,45.1003094517055,6.48085238195591,62.4885844033622
Time to assemble • SRTM processing: • Approx. 1.5 weeks • Blue marble processing: • Approx. 3 days( + perhaps 2 days to establish pixel format) • Setting up map in Cadcorp SIS: • ½ day • Server configuration • ½ day (from base install of Win2K3 Server + IIS 6)
Time to assemble (2) • Landsat 5 mosaic (Madagascar only) • Time to copy from DVD to server: 2 days • Extract and uncompress tiles in AOI, import into ArcInfo: 1½ days • Merge tiles & export: 1 day • ASP Web page creation: • 1 day • Work carried out on-site, and remotely via SSH and X Windows display
Hardware • Most of the processing carried out on a Sun Sparc Ultra 60 machine, with a 20GB local disk and using a Linux-based RAID array. • Web server: Dell Precision 210 - Pentium 3, 497MHz, 512MB RAM, 60GB local disk. Currently, the virtual directory contains about 3.4GB of data.
Immediate plans • Landsat 5 mosaic not currently integrated with server • Pressure of time • File sizes require additional memory in server? • Switch to Deegree Open Source system • Requires data re-engineering • Store full pixel data, not JPEG compressed • Improve user interface • Demo interface at present
Conclusions • Main data sets only recently acquired, therefore work is in progress • Basic mechanisms are working well with temporary interface • Some server, software and data issues to solve • An initial system with full data access for Europe/Africa expected to be ready by WGISS 18