1 / 11

Diamond

Diamond. Status. 3 beamlines with users, 1 starting commissioning. datasets > 20 images 2007 - 856 total 2008 – 521 (first 3 months despite problems!) Total data = 5.5Tb some compressed some not…. Directory structure. Data is stored in /dls/$BEAMLINE/data/$YEAR/”proposal-visit”:

brit
Download Presentation

Diamond

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Diamond

  2. Status • 3 beamlines with users, 1 starting commissioning. • datasets > 20 images • 2007 - 856 total • 2008 – 521 (first 3 months despite problems!) • Total data = 5.5Tb some compressed some not….

  3. Directory structure • Data is stored in /dls/$BEAMLINE/data/$YEAR/”proposal-visit”: • Where proposal is proposal number and visit is session on any beamline • Read-only* for backup and archive reasons • Subdirectories of “proposal-visit” created automatically: • spool : used by other beamlines • *processing: writeable area to be used for processing data by user (no quota) • jpegs: contains jpeg and thumbnails of all images • $HOME: quota enabled areas • Other subdirectories can be created by the user through GDA • Future directories • processed: will contain results from any processing software automatically run by diamond • jpegs of the crystal and loop from the inline viewing system will need to be stored

  4. Archiving Progress • ISPyB • Testing of GDA automatically writing to ISPyB underway • DNA ISPyB integration still to do but should be based on work already done • CBF • Archive of CBF (ImgCIF) or Vendor data format yet to be decided • Reliable migration/conversion tools now available • Data backup and remote access • Testing is underway for automatic archive of data onto tape and download from data portal. (Project currently subject to delays due to STFC “changes”)

  5. Software on Beamlines • Software is being made available through the ‘module’ system • Data processing packages available include: • Mosflm • XDS • HKL2000 • D*trek being investigated

  6. Automated Data Collection • DNA • Version 1.1 of DNA has been successfully tested on I03 in ‘manual’ mode • This has not been made available to users or tested on I02 and I04 • EDNA • Diamond is currently committed to EDNA as a successor to DNA

  7. Automated data processing • On datasets with >20 images, testing is underway to use a cluster to: • Process the data using XIA2 in both 2D (Mosflm) and 3D (XDS). (thanks to G. Winter) • Other packages e.g. ELVES are being investigated. Suggestions welcome.

  8. Cluster environment for CCP4? Classic Cluster • 24 dual dual core nodes • Sun Grid Engine • Future replacement NVIDIA Tesla • Massively multithreaded SIMD Architecture. • Thousands of threads required for efficiency • Per £3500 1U rack mount unit • ≈ 400 times the calculation performance of a single core Intel processor (optimized external to DLS, Financial Monte Carlo) • 40 times the performance for data heavy analysis (relatively unoptimized internal DLS code, tomographic reconstruction)

  9. Multi user shared environment and CCP4 • Automatic updates • Software managers mailing list? Monolithic releases are nice but…. Users don’t have permissions…. • Old CCP4i • No shared projects – future looks good though • Permissions problems? • Coot-backup • No coherent plan for exploiting high performance computing? • But as there are ‘resources’ at a large site it is possible within certain constraints.

  10. CCP4 and Diamond • Diamond does not do much with CCP4 and CCP4 does not do much with Diamond • Some exceptions mostly in data processing. • Areas we could do more together • General software developments (parallelisation, tools, testing, releases….) • Some information exchange both ways. • Technique developments and automation. • Vehicle to the users through their data. • But can anyone be bothered?

  11. Beamline Scientists Thomas Sorensen, Liz Duke, Katherine McAuley, Dave Hall, Gwyndaf Evans, Ralf Flaig, Armin Wagner Support Scientists James Sandy, Mark Williams, (Alice Douangamath) Post Docs Mike Latchem, David Waterman Technicians David Butler, Mic Harding, Trevor Orpin, Adam Prescott, Adam Taylor Engineering Martin Burt, Jeff Preece, Jerry Nash Controls Andy Foster, James O’Hea Software Jun Aishima, Alun Ashton, Josephine Chan, Paul Gibbons, Karl Levik, Richard Woolliscroft Recruiting Beamline Scientists Support Scientists Industrial Liaison Scientist The Staff

More Related