100 likes | 196 Views
Simulation Production at UTD. Shuwei YE, UT-Dallas DOE Review, Nov. 17, 2004. Outline. SP5 SP6 migration Operation challenges UTD Production Preparation for SP6 SP7 migration. SP5 SP6 migration. Started in the end of Jan-2004
E N D
Simulation Production at UTD Shuwei YE, UT-Dallas DOE Review, Nov. 17, 2004
Outline • SP5 SP6 migration • Operation challenges • UTD Production • Preparation for SP6 SP7 migration
SP5 SP6 migration • Started in the end of Jan-2004 • Smooth in general owing to last experience • Big changes in production • Problem with merging and export
Big changes in SP6 • No evt database, in ROOT format • Cond/cfg database only • Automatic transfer and cleanup non-stop production • Replace bbftp with bbcp because of file fragments in bbftp
Trouble shooting in SP • Objy-NFS lives with disk automount: • Trouble with spmerge, spexport: solved by “LD_LIBRARY_PATH” • spmerge failure in August caused by a VERY RARE case: a node failed just before a job was done
Hardware challenges • Occasional corrupt disk and bad memory we have spare disk, but no spare memory • A/C failures: July 2004, September 2004 • Power outage: electrical infrastrure upgrade • Corrupt file system loss of database and useful scripts
Hardward Problems • A/C problem (addressed in Xinchou’s talk) • Old RAID problem (spare disks available) • Rare unexpected power outage (could damage databases)
UTD SP6 production Power Upgrade Official Report UTD is No. 2 in the world before August
UTD total production UTD total production • 180 Million SP6 events • 170 Million SP5 events • 70 Million SP4 events
SP7 preparation Major changes besides routine updates • Objy 7.2 8.0.9 (wait until SP7 is ready) • Operating System: RHEL or SL We did testing on SL and have passed SP6 validation.