1 / 6

FNAL Production Progress

FNAL Production Progress. Problems: RAID Array Caused 4/8 weeks of downtime during April and May Configuration issue: may have been problem with vendor documentation - so far no new failures. Problems: Database corruptions (Majority of following OK - please use and give feedback!)

karinr
Download Presentation

FNAL Production Progress

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FNAL Production Progress • Problems: RAID Array • Caused 4/8 weeks of downtime during April and May • Configuration issue: may have been problem with vendor documentation - so far no new failures. • Problems: Database corruptions (Majority of following OK - please use and give feedback!) • Known corrupt metadata in jm_w_lj, single pion • Suspected corrupt metadata in jm_hlt2030 • May have been caused by botched oocleanup and/or loss of jnl files. • Problems: Human Error • jm_tttau and jm_wjet produced in fresh OBJY federation with same owner names: metadata cannot be merged with old federation. For now, served at FNAL from distinct User Fed. • Do we really have to rerun a week’s worth of data to change the owner name ? This is silly. ( Greg Graham - FNAL )

  2. FNAL Production Progress (thru end of May) Stats can be viewed at http://computing .fnal.gov/cms/Monitor /cms_production.html Follow link to “Status of OODigis” ( Greg Graham - FNAL )

  3. FNAL Production Scripts • Now: CMSIM, writeHits, writeDigis • Soon: Pal Hidas’ ntuple maker • Production Parameter Categories: • Stationary: Those that don’t change over a production cycle (eg.- pileup definitions, other things that we put in template files right now, …) • Volatile: Those that change for every distinct job (eg.- dataset name, owner name, other things we control from the command line now, …) • Control: Those that change unpredictably in response to farm conditions ( eg.- OO_FD_HOST, PATH to control where output is physically written. ) • The distinction is also used to identify how parameters need to be stored or queried upon ( Greg Graham - FNAL )

  4. FNAL Production Scripts • User invokes the Declare script • If not generating data, a designated input area will be scanned to define input. • Stationary parameters are copied into an area where the scripts will be built • User invokes the Create script • Scripts are created and stored to process the object data • Volatile parameters are fixed at this time • User invokes the Submit script • The Submit script submits created jobs to batch • Once the created jobs are running, the control parameters are read in. ( Greg Graham - FNAL )

  5. ( Greg Graham - FNAL )

  6. FNAL Production Scripts • Seehttp://computing.fnal.gov/cms/Monitor/cms_production.htmlfor details. • In use at • FNAL, CERN, UCSD, Bristol (?) • Trying them out at • INFN, Wisconsin, Florida • We are constantly upgrading for portability and evolving in response to many helpful comments from many people. ( Greg Graham - FNAL )

More Related