1 / 11

Erice – December, 06-08, 2017

A customizable tool for selecting, merging, and analyzing EEE trending data sets from multiple telescopes. Solve data redundancy and loss issues efficiently.

kowalski
Download Presentation

Erice – December, 06-08, 2017

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Erice – December, 06-08, 2017 Liceo Scientifico Statale Piero Gobetti - Torino ADM: Automatic Data Miner: a customizable tool for selecting, merging and analysis of the EEE trending data sets Lorenzo Sibille – Lorenzo Mathis

  2. Data from the over 56 telescopes all over Italy are sent to CNAF-INFN, in Bologna, to allow track reconstruction so that all relevant information can be stored in a database to be later available for analysis.

  3. Data for schools are available in csv format at http://eee.centrofermi.it/monitor

  4. Choose the telescope Select the day(s) And download data in csv format

  5. Problems: daily DQM report contains redundant data; you have to erase superimposing periods Data overlapping: Data loss: sometimes data are lost due to telescope malfunction or telescope switched off Result: time waste in choosing another data period and long data formatting time

  6. Solution: Automatic Data Miner - choose telescope and data period • concatenates many csv data file from chosen telescope and period - erase superimposing periods - make data consistency check (data loss) • output in csv, MySQL, HTML or Excel™ format

  7. Automatic Data Miner Web Interface

  8. Automatic Data Miner Web Server (PHP reads query) HTML5 + CSS3 + JS(JQuery) AJAX Post() Python a) new_user b) downloader c) data_unit d) data_check CNAF-INFN csv database a) Creates a /tmp/usr_x directory b) Downloads and concatenates .csv files c) Deletes superimposing records d) For each record checksiftimestamp(n+1)-timestamp(n) > MAX_DATA_LOSS

  9. Automatic Data Miner Lost data: 86 min Lost data: 73 min Lost data: 73 min

  10. Further implementations FATA: Forbush Automated Tool for Analysis Forbush decrease (07-16-2017) We’ll publish ADM on our website and we’ll make ADM available to CNAF-INFN EEE Monitor web site FATA will be developed also looking at future Centro Fermi - EEE (TO group) collaboration with INAF-OATO (Osservatorio Astronomico di Torino) Space Weather project

  11. Many thanks to: Dr. Ivan Gnesi - Centro Fermi Dr. Fabrizio Coccetti - Centro Fermi Dr. Alessandro Bemporad – OATO And many thanks to everybody at Centro Fermi!! Questions and info: Fulvio Poglio, f.poglio@liceogobetti.it

More Related