1 / 17

Data Filtering and Software

Data Filtering and Software. IceCube Collaboration Meeting Monday, March 21, 2005. Overview. Data Filtering and Software includes: Processing and Filtering (PnF) System at Pole Production Processing software environment Offline software suite IceTray software framework

tobias
Download Presentation

Data Filtering and Software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Filtering and Software IceCube Collaboration Meeting Monday, March 21, 2005

  2. Overview • Data Filtering and Software includes: • Processing and Filtering (PnF) System at Pole • Production Processing software environment • Offline software suite • IceTray software framework • Dataclasses event containers • Build system and tools • Database design and implementation • Database server • Access tools • Data visualization • Event Viewer • Root-based analysis tools Data Filtering and Software – E. Blaufuss

  3. DAQ Dispatch P&F Server Filt/Rec Data Raw Data Recon Worker X 50-60? PnF System Overview Expt. Control / Logging DAQ Land 4-15MB/s 1500-2000Hz ALL: RAW+FILT+REC FILT:RAW+FILT+REC Data Handling ALL:FILT+REC Transient Buffer Online Analysis Rec/Filt Data Online Analysis Tape Online Analysis Recon Monitor Offline Software/IceTray Data Filtering and Software – E. Blaufuss

  4. PnF system at Pole • V1.0 of PnF system was deployed at Pole this past season. • Running IceTray in Online environment • Parses and decodes the incoming DAQ stream • Currently performs no reconstruction. • Currently not in the Main data stream. • Lack of global run information at Pole • Lack of system-wide integration testing • No PnF output sent via satellite • TestDAQ/Monolith output tarfile only output sent • New version of Monolith soon (April) • Real testing of data flow to PnF (SPTS -> SPS) Data Filtering and Software – E. Blaufuss

  5. PnF Plans for next year • With the “Real DAQ” in place: • PnF output IS the output stream for RAW event data from IceCube DAQ. • All other non-event data will be handled by databases. • After this next deployment season: • Filtering at South Pole will be REQUIRED. • Will require reconstruction/filtering modules be ready and fully tested in IceTray offline! Data Filtering and Software – E. Blaufuss

  6. Offline Software • Several incremental releases this past year • In use by reconstruction and simulation groups. • Current release (new!): • OFFLINE-SOFTWARE V01-05-00 • Full details on release, and HEAD at: • http://glacier.lbl.gov/ • Plans for more functionality and tools over the next year. • More details regarding “readiness” of software later…. Data Filtering and Software – E. Blaufuss

  7. Databases (Mons group) • DOM assembly and testing database setup and in use for FAT testing. • Geometry database for String 21/IceTop up to date with Kurt’s latest numbers. • Includes “nominal” locations for other strings • Preliminary version of DOM calibration DB • IceTray module for DB access in place • In the next few months: • Run configuration DB for data from current year • Continued integration of database into IceTray Data Filtering and Software – E. Blaufuss

  8. Visualization • Event viewer in place • Animations • Fits, features, readouts • Root analysis Tree tools in place • Fit/reconstruction level analyses • Waveforms, features, hits level analyses • Plans for more over the next year. Data Filtering and Software – E. Blaufuss

  9. Current State of Offline SW(Is it “analysis ready”?) • IceTray software Framework • Has been released for ~1 year. • Can handle single events in a linear chain of modules • More advanced module topologies planned this year • Dataclasses event storage classes • Have been released for >9 months. • Designed for storage of • IceTray DOM data • Amanda data (treat as stand-alone/together) • Reconstruction results • Geometry, calibration, monitoring structures, more… Data Filtering and Software – E. Blaufuss

  10. State of Software (cont’d) • More extensions planned as need dictates. • Can hold TestDAQ data from this year • I/O routines for IceCube data • Monolith-reader can decode data from Pole. • Triggered events from South Pole in dataclasses • Basic functionality is there. • Includes domcal information (Thanks Dima…) • Still some work to do (trigger ID, run ID, etc) • Run “meta-data” – Run configuration info. • Data exists (files), but documentation and wider understanding is minimal. • Need docs, fill run configuration database to make properly available for analysis in IceTray. Data Filtering and Software – E. Blaufuss

  11. State of Software (cont’d) • Cleaning, feature extraction • Basic functionality in place and working • Hit selection/cleaning • FeatureExtractor (Dima) • Advanced cleaning needs to be implemented • Amanda specific: Cross talk, flare checking, etc. • Plans for that in next few weeks with Peter Steffen and others • Reconstruction • Basic algorithms are done (and reviewed!) • First guess (linefit, dipole, c-first, TOI, JAMS) • Standard Log-likelihood for muon tracks • Advanced (exotic PDFs, iterative fits) needed Data Filtering and Software – E. Blaufuss

  12. State of Software (cont’d) • Cut parameters/observables • Work has just started here and much remains. • Filtering capabilities • Currently possible to use linear chain of modules to reduce data files. • Branching of flow in modules planned for this year. • Allow muon and cascade recos in same chain. • Needs a “decision” module • Amanda • Full support for calibrated F2K files • Includes TWR in F2K files Data Filtering and Software – E. Blaufuss

  13. State of Software (cont’d) • TWR readouts (non-F2K) support this year • Working towards being ready to merge AMANDA/IceCube data this summer. • IceCube and Amanda data are both handled naturally together in IceTray/dataclasses. • Simulation (Significant driving force in the design and requirements for IceTray.) • Merging input frames to overlay events • Currently developing simulation releases against OFFLINE-SOFTWARE releases. Data Filtering and Software – E. Blaufuss

  14. String 21 Data – A few plots Data Filtering and Software – E. Blaufuss

  15. String 21 Data – A few plots More like this in Gary’s talk Data Filtering and Software – E. Blaufuss

  16. Status Summary Data Filtering and Software – E. Blaufuss

  17. Status Summary (2) • IceCube Offline software effort needs more: • Users • Developers of applications • Feedback, bugs found, documentation readers, etc. • We’re providing the framework and some tools. • Real work of providing reconstructions, analyses, simulations will be the work of the entire collaboration. • Start using it. It’s ready. • There are many ways to help out in software. Data Filtering and Software – E. Blaufuss

More Related