1 / 2

Archiver Nick West

Archiver Nick West. Status – as report in April Python based. Multiple client, multiple channel, fault tolerant archival. Uses PULL model: KEK pulls data from remote site No KEK access problems: only requires that KEK can ssh to remote site. Progress since April Successful soak test

eros
Download Presentation

Archiver Nick West

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ArchiverNick West • Status – as report in April • Python based. • Multiple client, multiple channel, fault tolerant archival. • Uses PULL model: KEK pulls data from remote site • No KEK access problems: only requires that KEK can ssh to remote site. Progress since April • Successful soak test • Ran for ~2 months and copied ~17K files Oxford -> KEK Datastore. • JLAN Access! • Finally emerged from registration process. • Still some rough edges – so far only works with IE with Sys. Admin. privileges. • Can login to t2k-login.j-parc.jp at Tokai. • Crude scp test to KEKCC shows single channel 500GB/day • Far in excess of estimated requirement: mean 30GB/day, peak 100GB/day • KEK Datastore access problem • PULL model won’t work (KEK cannot access JLAN). • But model could still be useful e.g. archiving during beam tests • Switch to PUSH (which is better – simpler). • Requires access from JLAN to KEK Datastore • Not properly working yet – can connect and select directory but not transfer data. T2K UK Collaboration Meeting

  2. ArchiverNick West • Remaining Tasks • Follow up problems accessing Datastore from JLAN. • Minor update to PUSH (exists but only writes to disk not Datastore). • Add GarbageCollector – to removes files from disk once archived. • Add WatchDog • Monitors Archiver and WatchDog and reports status to MIDAS. • Finalise database access and table design • Currently writes to simple table in local MySQL database. T2K UK Collaboration Meeting

More Related