100 likes | 188 Views
Types of Data. Points: Occurrences, Surveys Polygons: Census, Soils, Refuges Polylines: ? Rasters: Remotely Sensed, Models Volumes: Marine data 2D + Time: Climate (PRISM) 4D (3D + Time): Climate, Currents. marineemlab.ucsd.edu. Problems.
E N D
Types of Data • Points: Occurrences, Surveys • Polygons: Census, Soils, Refuges • Polylines: ? • Rasters: Remotely Sensed, Models • Volumes: • Marine data • 2D + Time: • Climate (PRISM) • 4D (3D + Time): • Climate, Currents marineemlab.ucsd.edu
Problems • Software/methods do not all support large datasets • Performance (i.e. time to develop methods and get final results) • Need to “reduce” the size of the data while maintaining the important information • Or, get a lot of computers • (more on this later)
File Formats • CSV, Txt: Points • Shapefiles • GeoDatabases • “Las” for LiDAR • HDF and NetCDF: • General hierarchical data formats • “CF” standard for NetCDF data • ArcGIS supports NetCDF
Data Reduction • Point Methods: • Clusters: group related data (spatially, temporally, categorically) • Gridding: find density, mean values • Windowing: moving a “window” over the data (does not reduce processing)
Polygons • Generalization/Simplification • Reduce resolution • Remove less critical polygons Soil Data for Czech Republic, eusoils.jrc.ec.europa.eu
Temporal • Group by: • Month, Season, Decade • Model “trends”
Software • ArcGIS will work up to a point • Then, we have to program • Python: • TXT and CSV files • Maybe for rasters, ND data • Java: • Effectively no limits • High performance
Databases • The simpler the data is, the faster it is to access: • Small, simple: • Text files • Small to Medium, complicated: • SQL Databases • Large: • Text and binary files • Avoid large, complicated data
BlueSpray • Java-based GIS application • Requires Java 7 • Built to be: • High-performance • Extensible • Portable • Takes advantage of RAM, processors • Easy to install and use • Owned by SchoonerTurtles, Inc. • Available at www.schoonerturtles.com • In early beta
Graphics 16 15 10 12 23 27 14 19 29 30 18 22 34 32 21 25