1 / 33

Raster data in GeoServer and GeoTools : achievements , issues and future developments

Raster data in GeoServer and GeoTools : achievements , issues and future developments. Ing . Andrea Aime GeoSolutions. GeoSolutions. Founded in Italy in late 2006 Expertise Image Processing, GeoSpatial Data Fusion Java, Java Enterprise, C++, Python

scorpio
Download Presentation

Raster data in GeoServer and GeoTools : achievements , issues and future developments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Raster data in GeoServer and GeoTools: achievements, issues and future developments Ing. Andrea Aime GeoSolutions

  2. GeoSolutions • Founded in Italy in late 2006 • Expertise • Image Processing, GeoSpatial Data Fusion • Java, Java Enterprise, C++, Python • JPEG2000, JPIP, Advanced 2D visualization • Supporting/Developing FOSS4G projects • GeoServer, MapStore • GeoNetwork, GeoNode, Ckan • Clients • Public Agencies • Private Companies • http://www.geo-solutions.it FOSS4G 2015, Seoul 14th-19th September 2015

  3. The technology stack FOSS4G 2015, Seoul 14th-19th September 2015

  4. GeoServer raster processing WCS WMS WMTS WPS GeoTools ImageIO-Ext JAI-Ext JAITools JAI-ImageIO JAI GDALKakadu FOSS4G 2015, Seoul 14th-19th September 2015

  5. Java Advanced Imaging • Huge number of basic building operations • Tile-based, deferred execution model • Integrate Tile Caching • Easily Extensible • Natively Accelerated Code available for many operations • Supports multithreading • Development more or less stopped! Read Read Crop Scale Mosaic Clamp Cache FOSS4G 2015, Seoul 14th-19th September 2015

  6. JAI-EXT • Built on top of JAI interfaces • Replaces most common JAI operations • Open Source (Apache 2.0) • Pure Java and High Performance • Support for No-Data • Support for Processing ROI • Several fixes compared to JAI • Support for Band Masks JAI-Ext JAI https://github.com/geosolutions-it/jai-ext FOSS4G 2015, Seoul 14th-19th September 2015

  7. JAITools • High performance raster processing • ExtendsJAI with more operations • Open Source, BSD license • Low Level Operations, e.g. RangeLookup • Higher Level Operations • Vectorize, Contour • Raster algebra (Jiffle) https://github.com/mbedward/jaitools FOSS4G 2015, Seoul 14th-19th September 2015

  8. Jiffle • Raster algebra languagebased on JAITools • Can be exposedas WPS https://github.com/mbedward/jiffle FOSS4G 2015, Seoul 14th-19th September 2015

  9. JAI - ImageIO • SUN then Oracle Image I/O library • Partly ships with JDK partly as extension • JAI bridge for tile-based, deferred data reading • Easily Extensible • Natively Accelerated Code available for many formats • No source code for native (de)coders • Development more or less stopped! FOSS4G 2015, Seoul 14th-19th September 2015

  10. ImageIO-EXT • Extends, complements, replaces Oracle JAI ImageIO Library • Open Source, LGPL and BSD licenses • Plugins: • BigTiff*, MatFile 5, TurboJPEG • JP2000 (Kakadu) • Fast PNG encoder • GDAL integration: JPEG2000, MrSID, ECW, BigTIFF, ERDAS Image, HDF4, … ImageIO-Ext ImageIO https://github.com/geosolutions-it/imageio-ext FOSS4G 2015, Seoul 14th-19th September 2015

  11. GeoTools • Adds the geography on top of the imagery • Representation of georeferenced grids • Reading and writing georeferenced formats • Rendering (colormap, contrast stretch, hillshade, transform, …) • Reprojection • Operations in geographic space • Process API and implementations ready to be used by WPS http://www.geotools.org http://github.com/geotools/geotools FOSS4G 2015, Seoul 14th-19th September 2015

  12. GeoServer • Adds the network protocols on top of the georeferencedrasters: • WMS and WMTS to view • WCS to download • WPS to process http://www.geoserver.org http://github.com/geoserver/geoserver FOSS4G 2015, Seoul 14th-19th September 2015

  13. Recent Achievements: JAI-EXT FOSS4G 2015, Seoul 14th-19th September 2015

  14. JAI-EXT improvements • QA and performance tuning underway • Mosaic images in different color models • Positive reports are flowing in. From the user mailing list: … JAI-Ext is giving me a >12x speed increase w/ warp transforms and bicubic interpolation against GridCoverages with floating point data types. …. Actually in preparing my report to the team I crunched the numbers and we're actually seeing tiling throughput improvements of: NEAREST by 3xBICUBIC by 21xBILINEAR by 68x Given our raster ingest rates of 30 Gpbs, this is significant FOSS4G 2015, Seoul 14th-19th September 2015

  15. JAI-EXT integration • GeoTools 14.0 and GeoServer 2.8.0 • JAI-EXT available on demand for the moment, add this system variable: -Dorg.geotools.coverage.jaiext.enabled=true • Still have to work out a few integration issues, should become active by default in the next few months FOSS4G 2015, Seoul 14th-19th September 2015

  16. Benefits of JAI-EXT integration • End to end NODATA support (scientific data anyone?) • Region Of Interest (ROI, raster mask) full support in all operations • Heterogeneous mosaic support: mix images in gray 8 and 16 bit, RGB in the same mosaic • Bug fixes over JAI • Speedups in certain operations FOSS4G 2015, Seoul 14th-19th September 2015

  17. Recent Achievements: Raster data masking FOSS4G 2015, Seoul 14th-19th September 2015

  18. GeoTools Vector mask support • Sidecar shp/wkt/wkb file with the valid area • Or single shapefile for the whole mosaic • Trouble: the vector rescales linearly, the raster does by the unit (cannot have 1/2 of a pixel, you know?) FOSS4G 2015, Seoul 14th-19th September 2015

  19. GeoTools Raster mask support • Support for GDAL external and internal masks • Binary images telling where the data is actually valid • One mask per overview • Very important for data compressedwith lossy compressions • The mask can be compressed too (DEFLATE, non lossy) FOSS4G 2015, Seoul 14th-19th September 2015

  20. Recent Achievements: Metoc specific FOSS4G 2015, Seoul 14th-19th September 2015

  21. MetOC: support for 0/360 rasters • In Meteorology and Oceanography, rasters with longitude between 0 and 360 are common • This is the data crossing the dateline (not request) • Improved support for displaying and re-projecting them Dateline FOSS4G 2015, Seoul 14th-19th September 2015

  22. Support for custom NetCDF projections • Added support for reading custom projections out of NetCDF and GRIB files • Register them in: $GEOSERVER_DATA_DIR/user-projections/netcdf.projections.properties • Usual WKT syntax 971801=PROJCS[“lambert_conformal_conic_1SP”, GEOGCS[“unknown”, DATUM[“unknown”, SPHEROID[“unknown”, 6371229.0, 0.0]], PRIMEM[“Greenwich”, 0.0], UNIT[“degree”, 0.017453292519943295], AXIS[“Geodeticlongitude”, EAST], AXIS[“Geodeticlatitude”, NORTH]], PROJECTION[“Lambert_Conformal_Conic_1SP”], PARAMETER[“central_meridian”, -95.0], PARAMETER[“latitude_of_origin”, 25.0], PARAMETER[“scale_factor”, 1.0], PARAMETER[“false_easting”, 0.0], PARAMETER[“false_northing”, 0.0], UNIT[“m”, 1.0], AXIS[“Easting”, EAST], AXIS[“Northing”, NORTH], AUTHORITY[“EPSG”,”971801”]] FOSS4G 2015, Seoul 14th-19th September 2015

  23. Support for custom NetCDF projections FOSS4G 2015, Seoul 14th-19th September 2015

  24. WCS NetCDF output improvements • Support for CF standard variablenames • Custom data packing • UOM conversion • Attributes FOSS4G 2015, Seoul 14th-19th September 2015

  25. Recent Achievements: Odds and ends FOSS4G 2015, Seoul 14th-19th September 2015

  26. Raster advanced projection handling • Improved support for requests crossing the dateline • Improved support for difficult projections FOSS4G 2015, Seoul 14th-19th September 2015

  27. Contrast stretch enhancement • Extensions to SLD and CSS • Extra params to control the stretch algorithm • Setup to be extended in the future <sld:RasterSymbolizer> <sld:ContrastEnhancement> <sld:Normalize> <sld:VendorOptionname="algorithm"> StretchToMinimumMaximum/ClipToMinMaximum /ClipToZero </sld:VendorOption> <sld:VendorOptionname="minValue">50</sld:VendorOption> <sld:VendorOptionname="maxValue">800</sld:VendorOption> </sld:Normalize> </sld:ContrastEnhancement> </sld:RasterSymbolizer> http://docs.geoserver.org/latest/en/user/styling/sld-reference/rastersymbolizer.html#contrastenhancement FOSS4G 2015, Seoul 14th-19th September 2015

  28. Contrast stretch enhancement StretchToMinMax Clip FOSS4G 2015, Seoul 14th-19th September 2015

  29. What’s cooking FOSS4G 2015, Seoul 14th-19th September 2015

  30. Hillshading • The one SLD 1.0 feature we were still missing • Did not make the GT 14.x/GS 2.8.x feature freeze, may be backported later <RasterSymbolizer> <ColorMap> <ColorMapEntry color="#000000" quantity="-32636" opacity="0" /> <ColorMapEntry color="#000080" quantity="-20" /> <ColorMapEntry color="#00BD62" quantity="30" /> <ColorMapEntry color="#98FB98" quantity="200" /> <ColorMapEntry color="#FBFE80" quantity="400" /> <ColorMapEntry color="#EDA765" quantity="1800" /> <ColorMapEntry color="#D9925D" quantity="4000" /> <ColorMapEntry color="#FFFFFF" quantity="5000" /> </ColorMap> <ShadedRelief> <ReliefFactor>20</ReliefFactor> <BrightnessOnly>false</BrightnessOnly> </ShadedRelief> </RasterSymbolizer> FOSS4G 2015, Seoul 14th-19th September 2015

  31. Hillshading FOSS4G 2015, Seoul 14th-19th September 2015

  32. Full JAI replacement • JAI interfaces and base infrastructure are not yet replaceable • Yet, they are very old code, we would like to have a fresh start based on what we have in JAI-EXT • Trying to get funds to get there, anyone here interested in helping? FOSS4G 2015, Seoul 14th-19th September 2015

  33. That’s all folks! Questions? info@geo-solutions.it FOSS4G 2015, Seoul 14th-19th September 2015

More Related