1 / 13

ICAT Integration at ISIS

ICAT Integration at ISIS. Tom Griffin, ISIS Facility ICAT Developer Workshop The Cosener’s House, Abingdon August 2009 tom.griffin@stfc.ac.uk. ISIS. Neutrons and Muons Operating since 1984 1600 user base; ~200 staff Volume of data ~6TB ~5M datafiles (27 instruments, 500/hour)

shania
Download Presentation

ICAT Integration at ISIS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ICAT Integration at ISIS Tom Griffin, ISIS FacilityICAT Developer WorkshopThe Cosener’s House, Abingdon August 2009 tom.griffin@stfc.ac.uk

  2. ISIS • Neutrons and Muons • Operating since 1984 • 1600 user base; ~200 staff • Volume of data ~6TB • ~5M datafiles (27 instruments, 500/hour) • >6.7GB metadata, >33M rows

  3. Current Pre-Experiment

  4. Data.ISIS

  5. Data.ISIS • Main point of access for ISIS data • Maintains index of Run number -> Folder • GEM12345 • \\isis\inst$\NDXGEM\Instrument\data\cycle_01_1\GEM12345.raw • .txt .log .sav0 • Ranges • Zip files

  6. Data.ISISAuthentication User performs search via application e.g. Data Portal 1 7 Data.ISIS Search is executed in ICAT 2 10 Permitted results are returned to application 3 5 Results are displayed to the user 4 8 1 User performs request to download datafile, multiple datafiles or dataset 5 4 Data Portal ICAT creates http GET link and passes to back to user (routed through application)sessionId email (optional)fileId(s) or datasetId action (i.e. download, zip, compressed) 6 2 9 User clicks http link 3 6 7 ICAT API Data.ISIS call ICAT API to check permissions sessionId & datafileId(s) or datasetId 8 Return Exception on failure or DownloadObject on success - userId - array [filename, cycle, run number] 9 10 User gets their data!

  7. Current Pre-Experiment

  8. ‘To Be’ Pre-Experiment

  9. Problems • RB Numbers • Business System • Not released to users • Replace DataPortal with TopCat

  10. Related Projects • Grid MP – distributed PC network • Analysis and simulation • New portal required anyway -> • Single point of access for Grid MP • Use ISIS data in Grid jobs • Store results and process in ICAT Where to store data?

  11. Acknowledgments(who did what at ISIS) Freddie Akeroyd (Data.ISIS) Chris Moreton-Smith (ISIS DFS/FRS) Hannah Guest (Facilities Business System & ERAS) Matt Clarke (SECI modifications) Damian Flannery Stuart Ansell (writeRaw) Laurent Lerusse (NXIngest) Database Services, Roger Downing, FBU IT Service Desk (servers and hosting) Core ICAT – Glen, Shoaib, Michael..... ISIS Computing Group ISIS Data Analysis Group

  12. Questions...

More Related