1 / 13

Netlobs Manipulating Gridded Data in a Relational World Neil Stamps Technical Architect

Netlobs Manipulating Gridded Data in a Relational World Neil Stamps Technical Architect. Agenda. Introduction to Lost Wax The problem framed Oracle 10g capabilities NetLobs – NetCDF meets database. Introduction to Lost Wax. Server-side Systems J2EE / MS.NET SOA and Web Services

haru
Download Presentation

Netlobs Manipulating Gridded Data in a Relational World Neil Stamps Technical Architect

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Netlobs Manipulating Gridded Data in a Relational World Neil Stamps Technical Architect

  2. Agenda Introduction to Lost Wax The problem framed Oracle 10g capabilities NetLobs – NetCDF meets database

  3. Introduction to Lost Wax • Server-side Systems • J2EE / MS.NET • SOA and Web Services • Legacy integration / wrapping • GIS Mapping Solutions • Multi-agent Systems • Continual R&D • Roles based business analysis • Agent framework • Products (SPL) • Innovation projects Advanced Software Engineering • Mobile • Distributed computing • Java PDAs / phones

  4. The problem framed Internet Web Service SQL Forecast Data GADS DTI DEWS research project partners: Met Office, Reading University, BADC, BMT, IBM Provides web services to multiple domains GADS provides marine services Oracle target platform

  5. Oracle – Blob support Oracle 10g deployment platform Large object support Blob – max size (4GB –1)*block e.g. 32k block = 128TB max Clob, nClob – max size as per Blob Extension support Java, C extensions Java stored procedures Custom data types (cartridges) Remote symbolic debugging (JDeveloper)

  6. Oracle - Custom data types Provide encapsulation of attributes and methods Introduce OO capabilities into relational world Allow unstructured data to be queried Extensions to indexes allow efficient queries Nested tables provide collection capability

  7. NetLobs – NetCDF ‘SmartLobs’ Provides NetCDF file capability to Oracle Encapsulates data and meta-data in single type Physical implementation agnostic Automatic extraction and storage of meta-data Interrogate meta-data without blob enquiry Extraction over single or multiple Netlobs

  8. NetLobs – NetCDF ‘SmartLobs’ NetCDF 2.2 open source Java libraries NetLob wraps Oracle for NetCDF Files Extraction interfaces based upon current GADS requirements: Subset Reduction Concatenation Higher level interfaces to be layered over basic functionality

  9. Cartridge invocation PL/SQL interface maps to Java call (or C) Oracle instantiates NetLob object Object implements SQLData interface Blob pointer passed to Java, Random access provided via ‘internal’ JDBC

  10. NetLobs – Data ingestion Upload using Oracle SQL*Loader Upload in two–phase method Validation at Netlob creation System optimisation based upon once-only performance hit at extraction of meta-data Meta-data ‘chunks’ will facilitate query by value

  11. NetLobs – Performance Reference GADS system provides predictable, linear extraction performance NetLob cartridge aims to achieve similar performance characteristic over large data extractions Optimisation tailored to larger extractions

  12. Moving forward Storage and retrieval of rotated data Pluggable interpolation framework Offloading processing to GRID Enhanced meta-data to meet community needs Query by-value enhancements

  13. Any Questions? Neil.Stamps@Lostwax.com www.LostWax.com

More Related