1 / 11

DØ RunI PDB Migration

DØ RunI PDB Migration. DØ RunI Production Database: The main purpose of DØ PDB is to keep the history of data processing by DØ Offline software and the relevant history of the raw data from which it came. It is also a central repository for luminosity information relevant to the data.

samara
Download Presentation

DØ RunI PDB Migration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DØ RunI PDB Migration DØ RunI Production Database: The main purpose of DØ PDB is to keep the history of data processing by DØ Offline software and the relevant history of the raw data from which it came. It is also a central repository for luminosity information relevant to the data. Current PDB is a relational database using VAX RDB environment. It has a number of utility programs written in FORTRAN language with embedded SQL commands. It has totally 57 tables, including: RUN, FILTER, TRIGGER, and tables for 13 file types. PDB migration: Move the current PDB from RDB system in the VAX machines to an Oracle database in SGI machines. Chunhui Han chunhuih@fnal.gov

  2. Chunhui Han chunhuih@fnal.gov

  3. 13 File types in DØ PDB • RAW Data • Reconstructed Data • STA DST • Streamed Data Files • STRM_STA STRM_DST • Filtered Data Files • FILT_STA FILT_DST • Micro Data Files • MSTA MDST • Streamed Micro Data Files • STRM_MSTA STRM_MDST • Filtered Micro Data Files • FILT_MSTA FILT_MDST Chunhui Han chunhuih@fnal.gov

  4. Comparison of the old and the new • Reason to use Python as the programming language: • A script language, easy to learn and debug. It has a vast array of modules to realize different functionalities. • Accessible on SGI machines at DØ. The module ‘DCOracle’ is installed. It provides easy access to the Oracle database from Python. Chunhui Han chunhuih@fnal.gov

  5. Old PDB Utilities • Existing utilities for the old PDB: • Miscellaneous Utilities: • GEN_TO_VMS PDB_LIST_FILES LIST_GLOBAL_RUNS • GET_RUN_TIME PDB_RUNFILES PDB_TRIGGERS • PDB_FIND_STA PDB_FIND_RAW PDB_PART_TAPE • PDB_TAPES_GT PDB_TAPEFILES PDB_TAPEFILES_GT • PDB_REPORT RECO_STATS RECO_VER_STATS • Luminosity Utilities: • PDB_FILE_LUM GET_FILE_LUM GET_FILT_LUM • GET_TRIG_LUM GET_RUN_INFO GET_RUN_LUM • RUN_FILT_LUM RUN_TRIG_LUM STRM_STA_REPORT • DB_DISPLAY Chunhui Han chunhuih@fnal.gov

  6. Current Status of the PDB migration • Oracle Production Database: • The new Oracle PDB is in the run_1_pdbtablespace in the development database: d0ofdev1. It will be moved to a permanent place sometime later, probable next week. • Now only those tables that are used by the new utilities are created, and populated with data (using Python scripts). That adds up to 46 tables. (11 tables less) It includes such key tables in the old pdb: RUN, FILTER, TRIGGER, and tables for 13 file types. Structure for each table remains the same. • PDB Utilities: • Most of the miscellaneous utilities in the old PDB are not useful. So I put them in the lowest priority, and concentrate on the luminosity utilities. To keep consistency, I maintain each program’s functionality the same as the old. Chunhui Han chunhuih@fnal.gov

  7. Current status of the utility migration • Luminosity utilities: • get_file_lum: Reads in a file containing a list of data files and returns in a file the luminosity for each file. • get_filt_lum: Given a list of filenames and a list of filters, calculate the luminosity for each file and for each filter. • get_trig_lum: Reads in a file containing a list of data files and also reads in a file containing a list of trigger names, returns in a file the trigger luminosity for each file. • get_run_info: Given a run number returns all the information about that run. • get_run_lum: Given a filename cotaining a list of run numbers, returns the corrected and minimum bias luminosity for each run. Also return the corrections for the corrected luminosity. • run_filt_lum: Given a file containing a list of run numbers and a se tof filters, returns the luminosity per run per filter for each filter in the set. • run_trig_lum: Given a file containing a list of run numbers and a list of triggers, returns the lumiosity for ech trigger for each run. Chunhui Han chunhuih@fnal.gov

  8. Misc utilities • Misc utilities: • list_global_runs: list all global runs in a given data range. • get_run_time: returns the start and end time of a given run or range of runs. • pdb_triggers: lists information about the triggers and filters for a given run. • pdb_runfiles: lists information about runfiles for certain runs taken between a range of time. • More utilities to be made if necessary. • Usage of the new utilities are made similar or identical to the old. Chunhui Han chunhuih@fnal.gov

  9. Output of get_run_lum • RUN NUMBER ACCLUM (nb-1) MB LUM (nb-1) LUM TOT(nb-1) MULT INT CORR MICRO BLANK CORR CENTZ CORR SINGZ CORR • ========== ============= ============= ============= ============= ================ ========== ========== • 60369 0.83470 0.69593 0.83470 1.21500 1.00000 0.35076 0.64455 • 60370 0.09343 0.16060 0.09343 1.20467 1.00000 0.34691 0.65119 • 60371 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 • 60372 14.98759 14.50127 14.98759 1.17716 0.94871 0.33805 0.66837 • 60373 12.79051 13.21778 12.79051 1.13125 0.91725 0.32411 0.69988 • 60374 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 • 60375 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 • 60376 11.64559 10.68744 11.64559 1.10623 0.91930 0.31661 0.71862 • 60377 3.11452 2.60446 3.11452 1.09249 0.94282 0.31277 0.72936 • 60379 2.95939 3.43760 2.95939 1.08374 0.91931 0.31031 0.73597 • 60380 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 • ========== ============= ============= ============= • TOTAL: 4.64257e+01 4.53051e+01 4.64257e+01 Chunhui Han chunhuih@fnal.gov

  10. Output of run_trig_lum RUN NUM. TRIGGER LUM (nb-1) ======== ======= ========== 60369 MISSING_ET 8.3470e-01 SCALAR_ET 4.3815e-01 60371 MISSING_ET 0.0000e+00 SCALAR_ET 0.0000e+00 60372 MISSING_ET 1.3946e+01 SCALAR_ET 7.8084e+00 60373 MISSING_ET 1.1732e+01 SCALAR_ET 7.1437e+00 60376 MISSING_ET 1.0706e+01 SCALAR_ET 6.8237e+00 60377 MISSING_ET 2.9364e+00 SCALAR_ET 1.9259e+00 TRIGGER TOT LUM (nb-1) ======= ============== MISSING_ET 4.0155e+01 SCALAR_ET 2.4140e+01 Chunhui Han chunhuih@fnal.gov

  11. Output of get_file_lum FILENAME ACCLUM (nb-1) MB LUM (nb-1) LUM TOT(nb-1) ======== ============= ============= ============= ALL_051271_06.X_STA01 0.00000e+00 1.33042e-01 1.33042e-01 ALL_051271_07.X_STA01 0.00000e+00 1.38232e-01 1.38232e-01 ALL_051271_08.X_STA01 0.00000e+00 1.38232e-01 1.38232e-01 ALL_051271_09.X_STA01 0.00000e+00 1.40591e-01 1.40591e-01 ALL_051271_10.X_STA01 0.00000e+00 1.36345e-01 1.36345e-01 ALL_051271_11.X_STA01 0.00000e+00 4.24603e-03 4.24603e-03 ALL_051380_01.X_STA01 0.00000e+00 4.31235e-01 4.31235e-01 ALL_051382_01.X_STA01 0.00000e+00 2.62238e-01 2.62238e-01 ALL_051382_02.X_STA01 0.00000e+00 2.11538e-01 2.11538e-01 ALL_051384_01.X_STA01 0.00000e+00 2.07628e-01 2.07628e-01 ALL_051384_02.X_STA01 0.00000e+00 2.14549e-01 2.14549e-01 Total Files processed: 11 TOTAL LUMINOSITY (nb-1): 0.00000e+00 2.01788e+00 2.01788e+00 Chunhui Han chunhuih@fnal.gov

More Related