1 / 27

NSO8055 Okeanograafiline prognoos Jüri Elken elken@phys.sea.ee

NSO8055 Okeanograafiline prognoos Jüri Elken elken@phys.sea.ee. Andmete haldamise küsimusi (väga lai temaatika, ühtne kontseptsioon puudub, siiski: http://www.iode.org ) ICES ajalooline andmehõive perfokaardi formaat (andmed) ROSCOP vorm (meta-andmed)

kevina
Download Presentation

NSO8055 Okeanograafiline prognoos Jüri Elken elken@phys.sea.ee

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NSO8055 Okeanograafiline prognoos Jüri Elken elken@phys.sea.ee Andmete haldamise küsimusi (väga lai temaatika, ühtne kontseptsioon puudub, siiski: http://www.iode.org) ICES ajalooline andmehõive perfokaardi formaat (andmed) ROSCOP vorm (meta-andmed) geograafiline kodeering (Mardseni kvadraadid) Andmete haldamise tendentsid MyOcean, Sea-SEARCH & SeaDataNet Meta-andmed: EDIOS

  2. 1881, Herman Hollerith Perfokaart IBM 80 column punch card format kasutusel alates 1928 kuni ca 1980 andmestruktuur kanti üle magnetlintidele numbrid 1 auk, tähed mitu auku kasutati ka “overpunch” FORTRAN reeglid põhinevad perfokaardile Card from a Fortran program: Z(1) = Y + W(1)

  3. ICES ajalooline andmehõive ICES Punch Card format (Data) 1968 ROSCOP Cruise Report format (MetaData) ICES perfokaardid Hydro Master Card iga jaama kohta 1, sisaldab meteoandmeid Hydrochemistry Card iga sügavuse kohta 1 kaart, sh riigi, laeva, koordinaatide, aja andmed Hydrography Card iga sügavuse kohta 1 kaart, kohandatud CTD-le (rohkem tüvenumbreid) andmete lahutus asukoht minut sügavus m T, S sajandik biogeenide suured kontsentratsioonid “overpunch” abil NB! puudub info metoodika ja kvaliteedikontrolli kohta

  4. ICES andmehõive kaasaegne juhend (ka HELCOM seire kasutab seda) Submitters of data should, if at all possible, use either of the formats described here. Receivers of data from ICES will receive it in only the first described format, the ICES Oceanographic 'punch card' format but software is normally supplied with any request to help the user read data sets prepared in this format, including an export facility to data bases and spreadsheets. The user should note that this format has been modified from that published by ICES in 1979 in several important respects. In particular provision was made to include position information to .01 of a degree, and time to the nearest minute. Other changes include a re-definition of the > (greater than) overpunch in the nutrient fields (type '56' chemistry record was replaced by type '76') and record type 'P6' was introduced to accommodate the very high nutrient levels reported from some coastal regions. In both of these record types chlorophyll 'a' is stored to only one decimal place (the '56' record type was 2 decimal places). From early 1994, additional features were added to the format (03 record) to accommodate extra decimal places common in CTD records. This affected only columns previously used for derived quantities (sigma-t, dynamic depth). For data received after ca 1997 parameters not supported by the above record types were included by the inclusion of the '0Z' record (Additional Parameter Record). This record type allows for any number of parameters, so long as it is specifified in the BODC/JGOFSdata dictionary. http://www.ices.dk/Ocean/formats/

  5. ICES perfokaardi näited: Hydro Master ja Hydrochemistry

  6. ROSCOP (Cruise Summary Report) ROSCOP (Report of Observations/Samples collected by Oceanographic Programmes) kinnitati IOC poolt 1960-ndate lõpus Ekspeditsioonide lühiinfo mida millega mõõdeti kus (Marsden kvadraadid) kelle käest küsida mõõtmisandmeid kaasajal analoog: meta-andmed Kirjelduste grupid: Meteorology (6) Physical Oceanography (18) Chemical Oceanography (16) Marine Contaminants/Pollution (8) Marine Biology/Fisheries (27) Marine Geology/Geophysics (16) Other (>30)

  7. Näide

  8. Data Tools and Models (SeaDataNet) Data. We make distinction between information and data. Information is describing a status or situation (e.g. today is cold); data quantify the status or situation (e.g. temperature is 10°C). We define data broadly to include experimental or in situ observations, model outputs and images. Visualization. The first significant examples of Marine Information Systems were developed during the '60s. Until the '80s the data management was composed of many independent processes: data collection; pre-processing; storage in files. Graphical representation and dissemination were part of the scientific study of the ocean dynamics. Today an efficient information strategy includes visual representation of data (graphs, maps, ...), as a tool for dissemination of data among users and the public. Quality control. There are areas were the use of term 'data' is often controversial: a) processed versus raw measurements, b) model outputs versus observations, c) images versus digital underpinnings. Scientist strive to fully characterise their data to enable a better understanding of its limitations. Use of data can be limited by the lack of certain attributes such as: procedures for collection, conditions during collection, instrumentation, temporal and spatial referencing, error or uncertainty, indications on quality assurance procedures. Analysis. Working with data provides opportunities for quantitative analysis and reasoning, broad discussion and debate to evolve scientific understanding. Models. We define as model an idealisation that embodies certain aspects of the 'real ocean'. Models provide an experimental apparatus for the scientific rationalisation of the ocean phenomena. In the presentation of ocean model fundamentals, it is useful to start with a discussion on fluid kinematics.

  9. Andmete haldamise tendentsid Kui andureid ja platvorme oli vähe, koguti andmed suurtesse keskustesse Kaasajal hoitakse andmed enamasti mõõtja juures, formaadid jms on kirjeldatud, ligipääs üle veebi, rahvuslikud andmekeskused on siiski tugevad: Meta-andmed kirjeldavad andmeid (parameetri definitsioon, metoodika, platvorm, kvaliteedikontroll jne) Andmete otsimine läbi meta-andmete kataloogi Andmeülekande protokollid (ftp, OpeNDAP jne) Operatiivne okeanograafia: mõõdetavaid parameetreid vähe, lihtsam suuri “süsteemide süsteeme” kokku panna Interdistsiplinaarne mereteadus: parameetreid tohutult, juba defineerimine keerukas

  10. Operatiivne süsteem

  11. Operatiivne süsteem

  12. Andmete “otsimise” projekt (juba lõppenud) http://www.sea-search.net/

  13. http://www.seadatanet.org SEADATANET has developed an efficient distributed Marine Data Management Infrastructure for the management of large and diverse sets of data deriving from in situ and remote observation of the seas and oceans. The on-line access to in-situ and remote sensing data, meta-data and products is provided through a unique portal interconnecting the interoperable node platforms constituted by the SeaDataNet data centres. The development and adoption of common communication standards and adapted technology ensure the platforms interoperability. The quality, compatibility and coherence of the data issuing from so many sources, is assured by the adoption of standardized methodologies for data checking, by dedicating part of the activities to training and preparation of synthesised regional and global statistical products from the most comprehensive in-situ and remote sensing data sets made available by the SeaDataNet partners.

  14. The marine observing system is highly fragmented: more than 600 scientific data collecting laboratories from governmental organizations and private industry have been identified. They collect data by using various sensors on board of research vessels, submarines, fixed and drifting platforms, airplanes and satellites, to measure physical, geophysical, geological, biological and chemical parameters, biological species etc. The collected data are neither easily accessible, nor standardized. They are not always validated and their security and availability have to be insure in the future.

  15. SeaDataNet üldine andmete haldamise kontseptsioon CDI Otsing 2005-2008 Andmeid on veel vähe! väljavõte 2010 http://www.seadatanet.org

  16. SeaDataNet produktid Soolsuse klimatoloogia 1975-2005 Jaanuar Aprill Juuli Oktoober Interpolatsioon: 4D Data-Interpolating Variational Analysis Soft: DIVA GHER (University of Liege) http://modb.oce.ulg.ac.be/projects/1/diva

  17. http://odv.awi.de/en/home/ • vabalt kasutatav • Windows (7, Vista, XP, 9x, Me, NT, 2000), Mac OS X, Linux, and UNIX (Solaris, Irix, AIX) • oma andmete formaat, kuid loeb ka NetCDF • rannajoon, sügavused

  18. Milliseid andmeid tuleb käsitleda? BODC = British Oceanographic Data Centre The BODC Parameter Dictionary In the 1980s, when we first started managing oceanographic data our dictionary contained less than twenty parameters. The BODC Parameter Dictionary is a collection of controlled vocabularies for parameter management. The BODC Parameter Usage Vocabulary (8 MB) contains almost 19,000 terms that are designed to label data values. These have been systematically constructed using a semantic model. Navigation through such a large number of parameters is a daunting task. To help with this, a 3-layer hierarchy of discovery keywords is provided. The top level is the SeaDataNet Parameter Disciplines, followed by the SeaDataNet Agreed Parameter Groups and the BODC Parameter Discovery Vocabulary. XML formaadis, kasutatav vastava tarkvaraga http://www.bodc.ac.uk/data/codes_and_formats/parameter_codes/

  19. SeaDataNet Parameter Disciplines asendatud veebisõnastikuga http://seadatanet.maris2.nl/v_bodc_vocab/welcome.aspx/

  20. SeaDataNet Agreed Parameter Groups asendatud veebisõnastikuga http://seadatanet.maris2.nl/v_bodc_vocab/welcome.aspx/

  21. EDIOS Meta-andmete formaat (MIF) (1) asendatud veebivormiga http://seadatanet.maris2.nl/v_edios/search.asp

  22. EDIOS Meta-andmete formaat (MIF) (2) asendatud veebivormiga http://seadatanet.maris2.nl/v_edios/search.asp

  23. Uued arengud (1) Marine Strategy Framework Directive foresees adoption of methodological standards for the assessment of the status of the marine environment, monitoring, environmental targets and the adoption of technical formats for the purposes of transmission and processing of data in line with INSPIRE Directive. In respect of each marine region or subregion, Member States shall make an initial assessment of their marine waters, taking account of existing data where available. Member States sharing a marine region or subregion shall draw up monitoring programmes and shall, in the interest of coherence and coordination, endeavour to ensure that: (a) monitoring methods are consistent across the marine region or subregion so as to facilitate comparability of monitoring results; (b) relevant transboundary impacts and transboundary features are taken into account.

  24. Uued arengud (2) EMODNET European Marine Observation and Data NETwork EMODNET will improve availability of high quality data. EMODNET will provide data on scales defined by the regions and subregions of the Marine Strategy Framework Directive. The parameters to be collated are chosen to fit in with the requirements of the Directive. Four service contracts were launched for creating pilot components: •Lot 1 – Hydrographic data •Lot 2 – Marine geological data •Lot 3 – Chemical data •Lot 4 – Biological data

  25. EMODNET vision: components

  26. Läänemere andmeid veebis Operatiivsed andmed (ainult näha)BOOS www.boos.org Ajaloolised andmebaasid ICES vabahttp://www.ices.dk/ocean/ BED vaba http://nest.su.se/models/bed.htm BALTEX parooliga http://www.gkss.de/baltex/data/index.html FMI ja SYKE (endine FIMR) SMHI (SHARK)

More Related