Concurrent Web Map Cache Server

1. Concurrent Web Map Cache Server Zao Liu, Marlon Pierce, Geoffrey Fox Community Grids Laboratory Indiana University

2. Introduction • Geographical Information Systems combine online dynamic maps and databases. • Many GIS software packages exist • GIS servers around state of Indiana • ESRI ArcIMS and ArcMap Server (Marion, Vanderburgh, Hancock, Kosciusco, Huntington, Tippecanoe) • Autodesk MapGuide (Hamilton, Hendricks, Monroe, Wayne) • WTH Mapserver™ Web Mapping Application (Fulton, Cass, Daviess, City of Huntingburg) based on several Open Source projects . • These are not compatible

3. Map Server Federation • Integrating GIS map servers is not trivial • Our solution: create a virtual map server to act as an agent server • Translates map requests from generic format to the format expected by the specific map server. • Provides a common language and programming interface for constructing clients • The agent server by itself will work but performance is not good • Must wait for slowest server to respond • Failure prone: a county server may not respond at all • Adds additional overhead for combining images • Combining the agent server with a caching server solves these problems. • Caches images for greater performance

4. Agent Server Architecture County Server Agent server

5. Caching Server • The agent server runs offline to harvest map images from county map servers. • Images are stored as tiles. • Tiles at county boundaries may be combined for greater storage and performance efficiency. • Clients connect to the cache server instead of the agent server. • The cache server constructs the requested image from pre-fetched tiles. • Inspired by Google Maps approach • Will enable more interactive clients (so-called AJAX programming) • Image construction may be parallelized/multi-threaded for greater performance. • Potentially takes advantage of new multi-core server architectures from Sun, Intel, and AMD.

6. Tiling Example Agent server requests entire county maps for a particular zoom level and then breaks up into tiles.

7. Tiling and Caching at County Boundaries Bounding box requests across boundaries have many empty tiles. Marion County Hancock County Removing these empty tiles decreases storage requirements and increases cache server performance

8. The combined map

9. Caching and Tiling Layers • Map servers typically contain base maps and optional layers • Parcel boundaries, roads, and township boundaries are layers. • We cache each layer separately. • Layers and base maps are combined dynamically using Java Advanced Image libraries. • Common techniques

10. Google Maps – WMS Integration • Currently Google Map supports attractive street map and aerial photo. • County map server supports more geographic information in details. • Issues -- Map Re-projection (Google Map and Sattellite are based on Mercator projection and WMS are based on EPSG:4326) -- Caching mechanism (should be the same as Google tiling method)

11. Map Re-projection Combine maps (no re-projection) VS Combine maps (re-projection)

12. Combine Google map with county parcel data

13. Tradeoffs of Caching • Cached images must be stored somewhere. • Currently, three counties (Hancock, Marion, and Cass) are cached at 11 different zoom levels. • Photo images, layers • Takes NN GB of storage

14. Caching the Entire State • Takes about 500-600 GB to store the entire state to zoom level 10 this way. • There are 6047804 tiles for zoom levels 0-10, 15131920 tiles for 0-11 levels • There are ~10 layers for each scale • Aerial photo layer tiles take 60 KB • Other layers (parcels, roads) are much smaller: 36 KB for all remaining 9 layers per tile • So we need 96KB * 6047804 tiles to store all map data • This is large but possible. • Current commercial servers hosts like Sun T2000 can have 1 TB external (RAID) storage. • We can easily spread our caching server over multiple hosts to store even higher magnification scales. • Efficient tiling storage can save disk space.

15. Summary of Contributions • Development of agent server to pre-fetch map images from county map servers. • Stores images as tiles. • Removes redundant/empty tiles. • Supports ESRI and OGC servers • Development of caching server • Provides a uniform mechanism for clients to interact with different map servers. • Increases performance and reliability • Don’t have to go to source map servers for every request. • Will enable more interactive clients • Google Map-like interaction

16. Next Steps • Add more counties • If county uses ESRI or OGC map server, current agent plugins can be used. • Formalize programming interface using Web Service standards • WSDL and SOAP • Investigate scalability and performance issues

17. Additional Slides

18. Results & demo • Performance improvement

19. Architecture CountyGisServer • XMLFetcher • MapFetcher XMLFetcher MapFetcher DB Buffer

20. Architecture – XMLFetcher • XMLFetcher • Request info from servers • fetches XML files not real data • Save xml into buffer for MapFetcher use • each represents a 1024 * 1024 image • Send request to several servers for a boundary map • With same coordinates

21. Architecture – XMLFetcher

22. Architecture – MapFetcher • MapFetcher • Query database • fetch real data from servers • Store maps and relate info on local disk • Tell DBManager to update the database

23. Tile Reconstruction • Projection Uniform • Layer overlap

24. Tile Reconstruction-- Example Parcel layer (combined with tiles)

25. Tile Reconstruction-- Example • Aerial photo layer