1 / 17

Experiences with Real-time Data Retrieval from Remote Observatories

Experiences with Real-time Data Retrieval from Remote Observatories using Iridium Communications Links. D.L. Detrick, T.J. Rosenberg, J.E. Etter, and L.F. Lutz University of Maryland Rick Sterling, Stephen Mende University of California, Berkeley Noel Petit Augsburg College.

elkan
Download Presentation

Experiences with Real-time Data Retrieval from Remote Observatories

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experiences with Real-time Data Retrieval from Remote Observatories using Iridium Communications Links D.L. Detrick, T.J. Rosenberg, J.E. Etter, and L.F. Lutz University of Maryland Rick Sterling, Stephen Mende University of California, Berkeley Noel Petit Augsburg College http://uap.umd.edu/Iridium

  2. Iridium Experiences ... Science/Observatory Overview: * PENGUIn/AGO program * AGO observatory * PENGUIn instrument bandwidths Hardware: * Iridium data system * Iridium/DAW/DAU DAS * DAW, Iridium chassis * CONUS Iridium chassis, antennas * System test, Berkeley, Nov. 2002 Software: * DAW firmware features * CONUS DAS features * CONUS DAS flowcharts Data Retrieval Results: * Iridium data throughput statistics * Iridium disconnections, resolution * Real-time access delays * On-line data access (static) * On-line interactive data access Summary: * Iridium link * Iridium disconnections * Data throughput

  3. PENGUIn: Polar Experiment Network for Geophysical Upper-atmosphere Investigations

  4. Data Retrieval and Distribution The data systems originally deployed with the AGOs, from January 1993 to January 1997, recorded data to on-site hardware, and the recorded media were retrieved during annual servicing visits. A newdata system was installed in December 2002, using Iridium communications links to a CONUS data acquisition computer, allowing real-time data retrieval and distribution. At the AGO, a data acquisition watchdog circuit (DAW) monitors the data flow, and is capable of autonomous action to remedy anticipated faults at either the data acquisition unit (DAU) or the Iridium modem. The CONUS PC software manages the continuous real-time retrieval of data records from each of the AGOs, and is similarly programmed for autonomous troubleshooting. Data are made available to researchers in real time, via an anonymous FTP server (ftp://space.augsburg.edu/pub/Iridium), and daily survey plots are published to another site, (ftp://antarcticdata.net/plots/). An interactive data plotting/retrieval site is nearing completion, which will allow interested investigators to select signals from any of the available observatories on demand, at selected time resolutions (http://uap.umd.edu/ArchiveData/).

  5. PENGUIn/AGO Iridium Data System The data acquisition unit at the AGOs is an important component of the new data system. The DAU manages the digitization of up to 24 analog signals, and the acquisition of data through up to 12 serial I/O ports (RS-232). The blocked data records are transmitted through a single serial output channel, with a bandwidth of 30-MB/day. Since this data rate is about 50% greater than the Iridium bandwidth, a 2-MB dual-port memory buffer was inserted between the DAU and the Iridium modem, and the data throughput is managed by a microcontroller supervisory circuit and the CONUS software. In addition to managing the data flow, the data acquisition watchdog (DAW) circuitry is programmed to recognize anticipated communication interruptions, and to perform actions to attempt correction of the problem. The CONUS data acquisition PC participates in the data management algorithm, and can likewise perform troubleshooting functions on the Iridium modem(s).

  6. AGO Iridium Data SystemBlock Diagram Watchdog Timer (2.5 Hr) 2 MByte SRAM Serial Data (RS-232) DAU Serial Data 8051 Embedded Controller 8051 Embedded Controller Dual-Port Memory Iridium Modem DAU Reset Pass-Through Mode 28 Volt Input Input Filter DC-DC Converter Output Filter 5 Volt Logic Power Control Power RF Data

  7. DAW/Iridium Data System Hardware

  8. CONUS Data Acquisition System The CONUS data system comprises a Wintel PC, with a 4-port serial I/O card, and a chassis containing three Iridium modems. The data acquisition software is a DOS/C++ program, running in a DOS window under Windows-95, or a Windows-NT variant. This platform made it possible to develop and thoroughly test the hardware/software in the available 3-month development interval.

  9. Iridium Data System Integration and Testing After a 3-month development period, the new Iridium data system was integrated and tested with the solar/wind power system that was to be installed simultaneously at three AGOs, during the 2002-2003 austral summer season. The testing hardware included a DAU, and the DAW/Iridium chassis. Data were recorded on-site, using a laptop computer, and also transmitted to the CONUS data acquisition system installed at the University of Maryland.

  10. Data Acquisition Watchdog Firmware The DAW firmware performs routine data management activities, and also is capable of troubleshooting hardware and communications problems. Activities: * Initialize modem * Buffer data from DAU * Transmit record to CONUS PC on demand * Pass-through command from CONUS PC to DAU * Process command from CONUS PC * Troubleshooting: - Reset watchdog timeout on receipt of valid command - Reset modem on receipt of 15 unknown commands - Reset DAU on demand - Reset buffer on demand Commands: * Send next data record * Re-send current data record * Reset buffer * Reset DAU * Status request * Enter/exit pass-through mode * Send firmware revision date * Generate simulated data on/off * Send buffer fill level

  11. CONUS Data Acquisition Computer Software Activities: * Initialize modem(s): - on bootup (all modems) * ‘Decommutate’ data stream from DAU: - concatenate data records for each channel - write hourly or daily data files * Issue a command for another record: - immediately on receipt of a record - at 3-minute intervals otherwise * Issue a command for the same record on checksum error; re-try up to 3 times * Update display, process keyboard commands * Get/display AGO/DAW status, every 15-minutes * Turn off VLF Snapshot channel, fill > 12 pages * Turn on VLF Snapshot channel, fill < 4 pages * Troubleshooting: - Re-initialize modem (single modem): > immediately on CD false transition (5 retries) > at 5-minute intervals while CD false DAW Commands: * Get next data record * Re-send current data record * Reset buffer * Reset DAU * Request status * Enter/exit pass-through mode * Get firmware revision date * Generate simulated data on/off * Get buffer fill level DAU Commands: * Turn VLF Snapshot channel on/off * Reset DAU clock

  12. CONUS Data Acquisition Computer Software Flowchart Iridium.cpp Iridium.ISR ftp://uap.umd.edu:2121/Iridium_DAS_Software/

  13. PENGUIn/AGO Iridium Data Acquisition During the period 23 December 2002 to 6 March 2003, over 1.5 gigabytes (GB) of data were returned from the three AGOs serviced during the 2002-2003 Austral Summer servicing season. The sustained data throughput was about 20-MB per day. Although there were frequent losses of signal between the paired Iridium modems, measurements of the data throughput indicated that the connections were capable of transferring 98% of the data bandwidth from these stations. Continuous data availability was achieved by cycling the data acquisition from the VLF Snapshot channel, by commanding the DAU to turn the channel on/off; these commands were generated autonomously by the CONUS data acquisition PC, using buffer full status from the on-site AGO DAW, and the DAU was commanded using the full-duplex link. During the 2003-2004 servicing, wind generators were installed at all three AGOs, enabling the acquisition of data into the Austral Winter. In addition, changes in sampling rates reduced the data throughput to about 15-MB/day per AGO; since this was within the Iridium throughput limit, data were available in real time, with no buffering delays. However, the available winds at the three sites were not sufficient to enable the wind generators to provide enough power to keep the electronics at operating temperature (>-30C), and all systems had performed controlled shutdowns by 4 April 2004. Since the start of data acquisition on 17 December 2003, over 3.5-GB of data were retrieved from the three sites.

  14. Survey Plots Survey plots are produced daily, and published to a web site. During the AGO servicing season, these plots are invaluable for helping the instrument PIs to evaluate the condition of the instruments, and daily Iridium phone communications allow troubleshooting tips to be passed to the servicing crew. The plots are also made available to the general public, so that interested investigators can examine the data. Dst < -470 nT

  15. Near-Real-Time Web-based Interactive Data Access D.L. Detrick and T.J. Rosenberg Fall 2002 AGU Conference Session SH03 http://uap.umd.edu/ArchiveData/ The technology needed for real-time interactive data acquisition and processing has advanced to the point that data even from remote observatories can be examined using common computers with access to the internet. The hardware on networked server computers can now store high time resolution data spanning years of observations, at minimal cost. Software developed for laboratory analysis of data can now be made available to the general community, using server software that permits dynamic access to the data and software resources. National Science Foundation sponsored research in the Arctic and Antarctic regions has produced an extensive database dating from January, 1982, from multiple observatories, and incorporating measurements from induction (searchcoil) and fluxgate magnetometers, bandpass filtered very-low-frequency (VLF) radio wave receivers, photometers, and multiple-frequency broadbeam and imaging riometers, at time resolutions as short as 0.1 s. Internet access to the remote data acquisition servers via satellite has enabled the database to be updated nearly in real-time. Plans are now being implemented to acquire additional data using Iridium communications channels, which permit actual real-time data access from multiple remote observatories. We describe a developing web-based system that permits data from this database to be plotted interactively using common web browsers, and which includes the capability for accessing processed data files in multiple formats that enables the data to be incorporated easily into other applications.

  16. Summary of Iridium Experiences • Iridium link: • - AT modem command language (modem-to-modem connection) • - Layer 2 networking protocol (data link: CONUS DAS<->DAW firmware) • - Fixed-length data frame (2053 bytes) • - Verified frame reception (sync word, checksum, re-transmission) • - No data loss due to transmission errors • Iridium disconnections: • - LOS ~4-5/hour, 2002/2003; ~15-20/day, 2003/2004 • - Autonomous re-dial on CD false status at CONUS DAS • - Modem re-initialization at AGO, triggered by garbled commands • (when idle, modem baud rate changes to ~3600 baud) • - 2.5-hour hardware watchdog reset (DAW, Iridium modem) • Data throughput: • - 20-MB/day (per Iridium channel) • - Achieved 98% of channel capacity, even with interruptions

More Related