1 / 32

EFW SOC Science Data Center (SDC)

EFW SOC Science Data Center (SDC). SOC-SDC Software Design Matt Born mattborn@ssl.berkeley.edu. SDC-RET: You Are Here. SDC-RET Data Retrieval Modules. Requirements: Retrieve data from external sources Deposit the data into an archive Log the transfer of data

alexis-woodard
Download Presentation

EFW SOC Science Data Center (SDC)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EFW SOCScience Data Center (SDC) SOC-SDC Software Design Matt Born mattborn@ssl.berkeley.edu RBSP-EFW SOC-CDR 26 Jan 2010

  2. SDC-RET: You Are Here RBSP-EFW SOC-CDR 26 Jan 2010

  3. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-RET Data Retrieval Modules • Requirements: • Retrieve data from external sources • Deposit the data into an archive • Log the transfer of data • Operate within fixed latency bounds Design principles: • Single “pull” interface per module • Single “push” interface per module, always to ARC-INT Implementation: • Python first draft • C optimizations where required • Most data transfers over SFTP per interface documents

  4. RBSP-EFW SOC-CDR 26 Jan 2010 SDC Data Retrieval Modules The recipe:

  5. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-RET Data Retrieval Modules

  6. SDC-ARC: You Are Here RBSP-EFW SOC-CDR 26 Jan 2010

  7. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-ARC Data Storage Modules • Requirements: • Archive data from internal and external sources • Log the source and time of deposited data • Maintain previous versions of deposited data Design principles: • No direct producer-to-consumer data transfer • Separate archives for public data versus private data Implementation: • Python first draft • C optimizations where required • At least HTTP and SSH access to archives

  8. SDC-PDP: You Are Here RBSP-EFW SOC-CDR 26 Jan 2010

  9. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-PDP Data Processing Modules • Requirements: • Synthesize new data products from existing ones • Log the sources and methods used for data creation • Deposit the data into an archive • Operate within fixed latency bounds Design principles: • No direct producer-to-consumer data transfer • Fully automatic processes with no user intervention Implementation: • Python first draft • C optimizations where required • Scripted execution of the NRT module for plot generation

  10. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-PDP Data Processing Modules The recipe:

  11. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-PDP Data Processing Modules

  12. RBSP-EFW SOC-CDR 26 Jan 2010 SOC-PDP Data Products

  13. RBSP-EFW SOC-CDR 26 Jan 2010 SOC-PDP Monitoring of Critical Telemetry

  14. RBSP-EFW SOC-CDR 26 Jan 2010 SOC-SDC Trending of Critical Telemetry

  15. SDC-MET<->UTC: You Are Here RBSP-EFW SOC-CDR 26 Jan 2010

  16. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-MET<->UTC Time Conversion Utility • Requirements: • Convert between times expressed in MET or UTC • Validate conversions against canonical MOC conversions Design principles: • Conversion speed is essential Implementation: • Python first draft • C optimizations where required • At least HTTP access to service

  17. SDC-MET<->UTC Time Conversion Utility RBSP-EFW SOC-CDR 26 Jan 2010

  18. SDC-NRT: You Are Here RBSP-EFW SOC-CDR 26 Jan 2010

  19. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-NRT Near Real Time Analysis Tool • Requirements: • Display waveforms and spectra • Display header-level information Design principles: • Leverage past project code base and IDL expertise • Provide an extensible and usable tool for future re-use Implementation: • IDL with Python extensions • Integration of legacy code • Scriptable operation

  20. SDC-NRT Near Real Time Analysis ToolInstrument I&T Usage RBSP-EFW SOC-CDR 26 Jan 2010

  21. SDC-NRT Near Real Time Analysis ToolSpacecraft I&T Usage RBSP-EFW SOC-CDR 26 Jan 2010

  22. SDC-NRT Near Real Time Analysis ToolFlight Usage RBSP-EFW SOC-CDR 26 Jan 2010

  23. SDC-BSEL: You Are Here RBSP-EFW SOC-CDR 26 Jan 2010

  24. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-BSEL Burst Selection Utility • Requirements: • Allow selection of burst data by timestamp • Log the selection of data • Automatically select data if user declines to do so Implementation: • Python first draft • C optimizations where required • Interface with CTG module for command generation

  25. SDC-BSEL: Burst Selection Utility RBSP-EFW SOC-CDR 26 Jan 2010

  26. SDC-BSEL: Burst Selection Utility RBSP-EFW SOC-CDR 26 Jan 2010

  27. SDC-BSEL: Burst Selection Utility RBSP-EFW SOC-CDR 26 Jan 2010

  28. SDC-DVAL: You Are Here RBSP-EFW SOC-CDR 26 Jan 2010

  29. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-DVAL Data Validation and Release • Requirements: • Display data eligible for release along with data necessary to decide upon release • Publish data deemed releasable • Operate within fixed latency bounds Design principles: • Reuse of existing modules for plot display • Strictly controlled route between private and public archive Implementation: • Python first draft • C optimizations where required

  30. SDC-DVAL Data Validation and Release RBSP-EFW SOC-CDR 26 Jan 2010

  31. SDC-SDT: You Are Here RBSP-EFW SOC-CDR 26 Jan 2010

  32. RBSP-EFW SOC-CDR 26 Jan 2010 SDC-SDT Science Data Tools • Requirements: • Display and analyze waveforms and spectra • Access data from the public archive, ARC-PUB We have determined that there exist numerous tools which will be applicable for scientific analysis of the instrument data.

More Related