90 likes | 180 Views
TeraGrid Archive Replication and Federation Phil Andrews et al. There is nothing more powerful than an idea whose time has come -Victor Hugo …system is in crisis, but nothing will be done before it enters chaos – C. Everett Koop. For those who came in late….
E N D
TeraGrid Archive Replication and Federation Phil Andrews et al There is nothing more powerful than an idea whose time has come -Victor Hugo …system is in crisis, but nothing will be done before it enters chaos – C. Everett Koop
For those who came in late… • In response to encouragement from Steve Meacham, we submitted a proposal, oriented towards Stimulus Funding, to provide a TG replication service. 6 Hardware partners, ~$12M total, primarily in first year. Providing for non-XD funded archives a major concern • Between submission and review, the Stimulus Funding possibility was removed • 16 (17?) member panel was divided on proposal, not recommended for funding. Will describe comments in this talk • Hope to resubmit, spread over 3-4 years, 2-3$M /year The best-laid schemes o' mice an 'men gang aft agley – Robert Burns TeraGrid Quarterly, Sep’09
Panel Comments on Proposal: • 5 “Excellent”, 8 “Very Good”, 2 “Good”, one thought a solicitation was essential • Lack of solicitation →significant confusion • Some thought it unnecessarily ambitious • Some thought it insufficiently ambitious A compromise is the art of dividing a cake in such a way that everyone believes he has the biggest piece. -Ludwig Erhard TeraGrid Quarterly, Sept’09
Proposal: Confusion and response • Two reviewers though it dependent on XD • Two didn’t understand SDSC’s position • Some didn’t understand why not solicited • Need to “set the scene” better • Need help from NSF The people may be made to follow a path of action, but they may not be made to understand it. -Confucius TeraGrid Quarterly, Sept’09
Proposal: too ambitious? • Reviewers didn’t like spending HDW $ early • Some didn’t think all data needed saving • Some didn’t think any data needed saving • Need to establish requirements (survey?) • Need to better justify data archiving New knowledge is the most valuable commodity on earth. The more truth we have to work with, the richer we become. -Kurt Vonnegut TeraGrid Quarterly, Sept’09
Proposal: not ambitious enough? • Reviewers didn’t think 4 years was enough • Concerns on software plan/value added • Need help from NSF (either more than 4 years, or why not) • Need more concrete software plan • Need more value added I have no spur to prick the sides of my intent, but only vaulting ambition… -Macbeth TeraGrid Quarterly, Sept’09
Proposal realignments: • No Stimulus $: no need for “front loading” • Less concern for non-XD funded archives at SDSC and NCSA, eliminates one motivation • Need a more strategic approach based on long-term HPC sustainability • What will be the future TG archival requirements and policies? • What do we want to be when we grow up? The unexamined life is not worth living - Socrates TeraGrid Quarterly, Sept’09
Is Archival Storage part of the TG mission? • Traditionally TG has been cycles oriented • Archival will take bigger slice of the future budget • Sites (prediction) will tend to “unload” archival • User Requirement for archive will not disappear • DataNet doesn’t seem to the answer • Archival must be separated from individual compute competitions to make full shrift • A concerted approach is in all our best interests If not us, who? If not now, when?”- J.F.K. TeraGrid Quarterly, Sept’09
Early NICS survey results TeraGrid Quarterly, Sep’09