Cosmic ray variations: yesterday, now days, tomorrow Virtual (distributed) Databases

1. Cosmic ray variations: yesterday, now days, tomorrow Virtual (distributed) Databases A. Belov, E. Eroshenko, V. Yanke

2. 1. Short overview the history of the NMN creation and CR data accumulation; • 2. Different small infrastructures’ distribution; examples of data usage –why we need so many stations? • 3. Existing archives and CR data bases. Modern tendencies of geophysical data base creating. • 4. Different ways of data presentation. Distributed data base for solar terrestrial physics; • 5. How to manage the work with distributed data bases? Possible approach- Virtual CR Database. • 6. Requirements TO and FOR distributed infrastructures. Accordance to the FP7 project

3. History of the neutron monitor network

4. Deployment of the neutron monitor network

5. World wide neutron monitor network

6. Presentation of NM data in the Internet in real time

7. All neutron monitors IGY (Simpson) NM – 5-6 now (mountains) NM64 Super NM 55 operating now 46 of them operatively present data 26- real time data

8. Sub infrastructures

9. DOURBES CR stations EC + Some words about forgotten stationDourbes • Dr. Sc. Hamid NEBDITel: 02/373 67 58E-mail: nebdi@oma.be Contact personnen Dr. Sc. Ir Jean-Claude JODOGNETel: 02/373 05 55E-mail: jodogne@oma.be Monitor 18NM64In 2002 year worked in Real Time. Resolution and updating 1-minutehttp://digisonde.oma.be/

10. DOURBES CR station - EC + DOURBES Neutron Monitor was situated in DOURBES, BELGIUMIt was a project of INSTITUT ROYAL METEOROLOGIQUE BRUXELLESLatitude: 50.10N Longitude: 4.60E Altitude: 225 m Rigidity (1965): 3.34 GVStandard atmospheric pressure: 1007 mb Barometric coefficient (1964): -0.71 %/mbThe mean count is around 780000/hour or 13000/min

11. Solar and galactic cosmic rays • There are two types of space phenomena for which monitoring and forecasting the CR data are very important. The first one is the strong space radiation storms caused by the solar cosmic rays accelerated to energy > 1 GeV during the great solar flares. The second one are the great interplanetary disturbances producing large geomagnetic storms in the Earth magnetosphere.

12. solar and galactic cosmic ray variations An example of the interplanetary disturbances created giant Forbush effects and severe magnetic storms in November 2004. GLE profile finished much earlier than max flux in low energies occurred

13. Exclusive GLE on 20 January 2005. Contour areas of equal fluxes of particles with rigidity>1 GV together with NM asymptotic viewing directions

14. One of applications of Space Ship Earth GLE radiation intensity in Earth’s polar regions in real time, obtained from 13 high-latitude (atmospheric cutoff) neutron monitors. Shown here is a sample map for the peak of the event of January 20, 2005. Note the extreme anisotropy in this event.

15. Precursor of the shock arrival in the CR variationsLongitudinal distribution of CR variations All stations Derived from European + Russian stations Derived only from European stations

16. Existing archives and data bases • Archives and data bases on GLE and 5/or 1-minute data • GLE data - Australian Antarctic Data Centerhttp://aadc-maps.aad.gov.au/aadc/gle/index.cfm (58). • Archive of GLE data on the serverftp://cr0.izmiran.ru/COSRAY!/FTP_GLE/ (70). • Archive of the minute data on the server ftp://cr0.izmiran.rssi.ru/COSRAY!/FTP_NM1/ (3800 monthly writing). • But mainly, 1- and 5- minute data are stored on the station (or small center) servers, for example, SpeceShipEarth data-on the server of Bartol Reseach Institutehttp://neutronm.bartol.udel.edu/.

17. Hourly, monthly, yearly NM DATA WDC-A (Boulder, Colorado, USA) ftp://ftp.ngdc.noaa.gov/STP/SOLAR_DATA/COSMIC_RAYS/ WDC-B (Moscow, Russia) http://www.wdcb.ru/stp/data/cosmic.ray/ WDC-C Ibaraki, Japan http://www.env.sci.ibaraki.ac.jp/database/html/WDCCR/data_e.html The most complete data base (since 1953 from 98 st) is on the IZMIRAN serverhttp://cr0.izmiran.ru/common/links.htm Data from individual stations are accumulated at their servers or at separate small Centers (Bartol, Jungfraujoch, Rome, South Africa).

18. Existing database in IZMIRAN • Data from20 stations are updaded in real time (every hour) and accessible in graphical and digital form. 59 stations

19. The present idea: not accumulate but integrate! • The volume of scientific data during last years leaves less and less possibility and necessity to accumulate at one place all data from different remote sources. • The orientation is changing to a direct usage data from remote archives or real time measurements by different detectors. • With this respect each station should arrange real time publication high quality data providing easy and free access to them. This will allow a distributed data base of small infrastructures to be designed for wide scientific usage.

20. Data publication in real time How to make real time data publication to be optimal? How to arrange remote access by one of standard ways (FTP, HTTP)? The first advise – to make it maximum simple. But this simplicity is very often absent! The simplest way to access data is transferring them by FTP. Very successful example is data service in GOES project. Minute data are stored in daily files and, besides, the current file with updating of 1- minute data is supported every last two hours. (http://www.sec.noaa.gov/ftpdir/lists/xray/). __ It allows take remote data and work with them: http://cr20.izmiran.ru/SW/main.htm __

21. Real time data publication The user applies to necessary files accordingly its names. Applying to the current file gives data for the last 120 minutes. Except of advantages it contains some drawbacks, since under minute request of data it is necessary to transmit all current file that increases traffic in 120. The size of current file has to be chosen on the compromise basis: maximum size is limited by traffic, minimum – by the ability of the system to recover data in case of failure connection during some time. In this sense the current file as a data for the last minute or monthly file of minute data are the examples of quite unlucky solution.

22. Real time data publication Much more flexible is a request by HTTP. Data of any capacity are transferring with the same efficiency, that provides a possibility to take data of necessary volume. However, in this case the requirements to data structure are higher: if in the first way (FTP) data may store in simple archive then in the case of HTTP they have to be arranged into data base and system make a selection respectively to a request. The samples of good organized http publications: http://cosmicrays.oulu.fi/ Orhttp://pgi.kolasc.net.ru/CosmicRay/ Or http://cr0.izmiran.rssi.ru/mosc/main.htm; http://cosray.phys.uoa.gr/; http://134.245.132.179/kiel/main.htm

23. Real time data publication Sample of unsuccessfully elaborated database: it takes 12 sec to take data for 1 minute . The whole table is transferred, by request, but it contains only 5% of useful information. File structure of GOES project

24. Real time data publication

25. How to unite? “Virtual System” • Despite of different data presentation we have a chance to integrate all these infrastructures into one “Virtual Cosmic RayObservatory”. “Virtual Observatory" is based on the system of WEB technologies to search for and access data, remote processing and visualization of data distributed by the remote independent archives. • Elaboration and creation of distributed information systems has essential deployment during the last 2-3 years. IVOA (Intern. Virtual Observatory Association) was established in 2002 and start of complete deployment of VO functioning is planned on 2007-2008. • Virtual Observatory allows maximally simplify searching and access to the data of different origin, make their preliminary viewing, selection, processing and analysis.

26. Virtual Observatory • For example, let us go to the Virtual telescope SkyViewand direct it to a definite part of the sky with coordinate +23.0 +13.0. • http://skyview.gsfc.nasa.gov/ . In a moment you see excellent picture. This Virtual Observatory (VO), may present picture obtained yesterday, year before or in real time by the monstrous amount of data from different sources. But we should only to start with this creation.

27. “Virtual Observatory”and“VirtualArchive” • VO is based on METADATA – a description of data on the remote servers. IVOA standard of metadata presentation as VOTable, despite of some advantages is too complicated and has a narrow area of application– Astronomy. • With this respect in the project “Virtual Archive”in IZMIRAN initiated, the alternative version of metadata (VATable) was developed and used. We don’t need to get data in common format, we should well describe the format used at the station. • For metadata presentation a popular approach was accepted, closed to IVOA recommendations (http://www.ivoa.net) and based on usage ofthe XML standard.

28. Sketch of the Virtual CR ObservatoryHow can we use this approach?

29. How to implement a project? “Virtual Archive”? • NM Network as a Net of distributed remote infrastructures can be integrated into one Virtual CR Observatory. • Approach“Virtual Archive” is more focused on the maximum simple searching for and easy access to the data for further local processing. • It distinguishes by a minimum level of requirements to the remote archives; • Two main requirements: 1) organization of the archive suitable for a remote access by one of the standard ways (FTP, HTTP); 2)maintenance metadata in actual stage.

30. SUMMARY A World wide Net of cosmic ray stations is a steady structure successfully operating more than 50 years, which proved its vital capacity and necessity; This net has own infrastructure, history, but there is NO and has never been a united managing (or, if only coordinating) Center; Is it good or bad – not simple question. From one side we definitely need orderliness and integrating, but from other side the history is full of examples when superfluous centralization and super orderliness spoiled lively affair.

31. To implement the full or part integration of the world wide CR station network this net should correspond to some basically requirements. The project has to: • Promote further obtaining data at the stations and improving their quality.The people directly responsible for data obtaining have to be interested in the project fulfillment. • Promote an improving exchange by the data on the CR variations and facilitation of access to data (including real time data). • Increase an interest to CR variation data by the adequately serving cross-disciplinary needs. • Use the global character of the world wide net and focus on solving of the global objects; • e-Infrastructures has to foster the emergence of new working methods based on the shared use of resources across different disciplines.

32. THANK YOU

33. Creation of Virtual Laboratory” • Site Virtual Laboratory is created on the principals of “Virtual Archive”: (http://vlab.izmiran.ru) aimed for hosting of different projects, creation tools for modeling and distributed data processing. На сайте, в частности, размещен интерактивный web-ресурс адаптивной обработки экспериментальных данных, элемент виртуальной лаборатории обработки изображений: инструмент подготовки графических изображений Солнца из исходных файлов в формате FITS, полученных при помощи рентгеновского телескопа СРТ-К (ФИАН) в рамках осуществления проекта КОРОНАС-Ф (ИЗМИРАН).