1 / 27

PaPCo Support for Cluster

PaPCo Support for Cluster. PEACE/Rapid workshop Aug 24, 2006. What’s PaPCo?. IDL software for data visualization and analysis modular: papco modules built by various institutions and individuals plug into papco core to provide data products

maryrobinson
Download Presentation

PaPCo Support for Cluster

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PaPCo Support for Cluster PEACE/Rapid workshop Aug 24, 2006

  2. What’s PaPCo? • IDL software for data visualization and analysis • modular: papco modules built by various institutions and individuals plug into papco core to provide data products • Modules provide graphic panels that are stacked on a time axis • ~60+ modules exist, ~30 are supported as “core modules” that come with papco. • Modules from 15+ spacecraft, including CRRES,Polar,Cluster. (and CDAWeb) • open-source, hacking encouraged. (the core is under CVS control)

  3. PaPCo main and draw widgets

  4. PaPCo History • ~1993: built for CRRES data analysis at Lindau • ~1996: generalized to support POLAR • ~2000: use for graphics server on Cassini • Supported at LANL • ~2004-present: support for Cluster

  5. Recent Efforts—clean up • Over time, developed a huge code base with lots of boiler-plate code. • Factor out common patterns in modules to reduce code volume and make behavior more robust and consistent • For example, retrieving data from FTP site and maintaining local database.

  6. Recent Efforts—core modules • Provide more uniform capabilities for core modules: • Retrieve data automatically • Support for Windows, Linux and Mac OS X • What-you-plot-is-what-you-get digital output • Useful out-of-the-box, no installation

  7. Recent Efforts—Cluster modules • PEACE module based on sddas • Cluster Prime Parameters • Cluster RAPID

  8. Recent Efforts—generally useful modules • OMNI • Papco-cdf cdaweb archive • Slices module

  9. Recent Efforts—PaPCo runtime • IDL Runtime is freely available—no license • We combed through papco code to support IDL runtime mode • Also fast loading and convenient. • Nice for web servers—fast, don’t waste license

  10. PaPCo Idioms • “Panel” is a region of the page painted by a module and editor • “Product” is a configuration file that describes panel layout and time range • “Configuration” is the set of loaded modules • Batch files can be used to control time axis for batch processing (command line supported)

  11. PaPCo Idioms • Factored out common patterns into IDL objects • Developers are still free to implement modules as they want, but objects provide common functions that can greatly simplify modules. • Certainly useful to module developers • Good for scientist-end user to be aware of what’s going on.

  12. Idioms--Filesystem • Filesystem objects access local or remote (via http, ftp, ssh) resources. • Filesystems are persistent to session, so remote changes aren’t seen unless there’s a papco restart or the fs is explicitly reset. • Remote listings are slow—we may change persistence. • Offline/Online support

  13. Idioms--File Cache • Set of local and remotely-available data files the module can get at. • Specified with a template: %Y%m%d_data.cdf • File cache tool provides list of available files, buttons to go get data • Module using a cache just asks the cache for the files that cover the requested time period • File cache is set of filesystem objects

  14. File Cache Tool

  15. Datasets Abstraction and Cache • Data structure for holding data of arbitrary rank (e.g. spectrogram=rank 2), with metadata • IDL structure similar to cdaweb cdf file conventions. • Papco provides methods for reducing and slicing data sets. • Dataset cache is RAM cache for storing loaded data • Visualization routines for datasets include lineplots, spectograms and radial spectrograms, and support data output automatically.

  16. Datasets Abstraction

  17. Display of loaded datasets

  18. Slicers • Slicers provide a view of the data at a given time within a greater context. • Generally reduce rank—spectrogram reduced to spectrum • “Papco12 compliant” slicers can embed slices on panel, supports output

  19. RAPID slices

  20. Peace Slices

  21. More idioms • Keychain manages credentials • Current output file template specifies where output goes: .png, .ps, .txt, .etc.

  22. PEACE module • Started with Matt Taylor’s module, reworked as proof-of-concept for dataset abstraction • Data comes from sddas’ IDL interface • DEF( time, energy, pitch ) (or counts, DNF) • Panel reduces dataset to energy-time, pitch-time • Slicer provides DEF(energy), DEF(energy,pitch) or distribution plot. (papco 12 compliant, so series of thumbnails and soon movies)

  23. RAPID module • Provides standard Rapid data products • Papco 12 Slicer support • 4SC products partially done • Gradiometer • RAVE (LASP viewer)

  24. Rapid and slices module

  25. PEACE and slices

  26. Cluster CDF Prime Parameters • Derived from general-purpose papco cdf module, optimized for cluster • Select instrument and “add dataset,” module goes off and downloads a cdf file and data sets are discovered

  27. To be done • Papco 12 is just today going into beta stage • Cluster Active Archive interface • PaPCo is end-user driven, each feature is there because someone wanted it—we want feedback • PaPCo soon to be on sourceforge.net

More Related