InGeoCloudS INspired GEOdata CLOUD Services

1. InGeoCloudSINspiredGEOdata CLOUD Services Linked Open Data for Data Services on the Cloud www.ingeoclouds.eu Dimitris Kotzinos University of CergyPontoise and FORTH-ICS This content by the InGeoCloudS consortium members is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Based on a work at http://www.ingeoclouds.eu/.

2. Big Data? • “Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone is doing it, so everyone claims they are doing it …” • Dan Ariely, Professor, Duke University

3. ID-Card of the ProjectObjectives in a nutshell • Design and build a cloud infrastructure for public agencies in the spatial-environmental field • Provide an innovate (flexible,…) infrastructure for geo-data services • “Move” public services to a cloud‐based infrastructure. • Integrate geo-data by exploiting a Linked Data “model” • The project wants to demonstrate that a Cloud infrastructure can be used by public organisations to provide more efficient, scalable and flexible services for creating, sharing and disseminating spatial environmental data

4. ID-Card of the ProjectPartnership • 5 Geological Surveys bringing in 6 initial Use Cases (datasets and applications) • Ground Water Management • Geo-Hazards • GeoData Publication and Mapping • 3 ICT organizations bringing key-expertise • Cloud Computing • Semantic Web and Linked Data • GIS • Software architecture and integration… • EC Support

5. ID-Card of the ProjectKey Dates Regular Exploitation Pilot 2 Pilot 1 Westarted Feb 2012 October 2013 March 2013 July 2014 MAY 2012: Experts Workshop#1 NOV.2013: Experts Workshop#2

6. ID-Card of the ProjectAchievements • Fundamental scalable/elastic services for data management: Database Server, File Server, Linked Data Store • Data publication modules • An API: Web Services upon a REST-based architecture • Data providers’ data and applications in the cloud • Portal and Management Tools

7. Infrastructure DescriptionThe Big Picture

8. IntegratedGeo Applications

9. MapDesign and Publication Service

10. From the map to the INSPIRE service • Upload my dataset • Create a new map (context) • Create Layers (from dataset) • Add functionalities • Publish the map • Finalize the map Tag mydataset Definesmy services Publish services Metadata View, Download Public Catalog

11. ID-Card of the Project(Big?) Data and the 5 Vs • 1 billion triples in the RDF triplestore published as Linked Open Data (LOD) • Datasets from 3 different scientific fields • Ground water (information and chemical analyses) (slow change) • Landslides (fast change) • Earthquakes (real-time measurements) • Datasets from 4 different countries • Denmark (Relational Data) • France (Relational Data) • Greece (text/xls data, XML data) • Slovenia (Relational Data) Volume Velocity Variety Veracity X Value Later

12. Linked Data Services GeoJSON KML ShapeFile …. tsv/csv xml json geoldtransform ldquery addXSLMappings geoldquery formats xml trig trix n3 … addR2RMLMappings ldexport Inspire compliant ldimport inspire_export/ inspire_query_export

14. QueryingLinked Data Store

15. GeoProcessing Geoprocessing implemented as a WPS service: refers to ordinary kriginginterpolation • WPS is provisioned in the ElasticWebServer component and uses parameters: • Given by the user • Fetched from InGeoCloudSTriplestore using LD-API

16. Monitoring and CostsMore Big Data? • A key needs in order to master one’s Cloud-based system. • Two main levels in InGeoCloudS that help in better managing the cloud: • Applications (analytics about usage) • Resources (supervision of indicators, alerts and problems management)

17. AuthorizationImplementation • Applications rely on a SSO token • Applicationscheck the user session and retrieve user profile • Applications buildpermissions fromthe user roles • To customize the behaviour of the application (e.g. mail on event), • To control access (e.g. to somedownload services), • To filterresultsdisplayed on the application GUI. • Each software component of the platform control access to itsownresources • Different user types/roles • Public (anonymous user) • Registered user • Data and Application provider • Administrator

18. INGEOCLOUDS *aaSofferings • PaaSElements • Authentication • Roles • Workspaces, • Portal • RESTful APIs • Basic Mapping Framework • IaaSElements • Servers • Network • Basic storage • Administration APIs • DaaSElements • GSOM • LOD APIs • Data storage, import, sync • Meta-data description, • SaaSElements • Geo applications, • Geo Publication, • Catalogues Management, • OpensearchQueries • Geo-Processing

19. Challenges related to Big Data and the Cloud“Political” Challenges • User adoption • Users are happy with what they have and they patiently wait until the problem surfaces • Users with weak or no infrastructure are most receptive to turn to Cloud and/or Linked Data paradigms than others • User see the cloud as a platform with more resources (memory, storage, processing) but they still want their applications to run there unchanged • Security, privacy and trust • Public vs. Private Clouds • Publicly owned vs. Company-owned Clouds

20. Challenges related to Big Data and the CloudScientific Challenges • Big Data storage • No “native” RDF triplestore that exploits the Cloud capabilities • Data security / privacy / control also a technical issue, especially when we deal with big data that you cannot monitor precisely • Querying Big Data • Fast querying for big data (many times the queries are simple but the volume is big) • Support for geospatial queries • (Fast) Indexing for (RDF) geospatial data • “Exchanging” Big Data • Data providers want to keep their own infrastructures and synchronize data between them and the cloud! Issues on synchronization and exchange of big data … • Monitoring and Management on the Cloud • More big data to manage?

21. Thank you and Questions www.ingeoclouds.eu Dimitrios.Kotzinos@u-cergy.fr