1 / 10

From Compute Intensive to Data Intensive Grid Computing

From Compute Intensive to Data Intensive Grid Computing. Welcome to the 2 nd Annual Condor meeting!!! (Hope to see you all in the EuroGlobus+Condor meeting in southern Italy in mid June). The Condor Project (Established ‘85).

jasonperry
Download Presentation

From Compute Intensive to Data Intensive Grid Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. From Compute Intensiveto Data Intensive Grid Computing

  2. Welcome to the 2nd Annual Condor meeting!!!(Hope to see you all in the EuroGlobus+Condor meeting in southern Italy in mid June)

  3. The Condor Project (Established ‘85) Distributed Computing research performed by a team of 26 faculty, full time staff and students who • face software engineering challenges in a UNIX/Linux/NT environment, • are involved in national and international collaborations, • actively interact with academic and commercial users, • maintain and support a large distributed production environment, • and educate and train students. Funding - DoD, DoE, NASA, NSF, AT&T,IBM, INTEL, Microsoft and the UW Graduate School

  4. State of the project • Extended the scope and size of the project • Extremely busy • Defended and preserved our unique approach to distributed/Grid computing • Enjoyed the impact we have on “real” users • Maintained the reputation of our research and the quality of our technology • Worked hard to make a commercial Condor fly alongside our project

  5. The Physicists & Us • Some of us have our roots in physics (Derek, Doug, Miron, Raj) • Long history of joint work (NIKHF 91,INFN 96, PPDG 99, SMC, WA92 COROUS, PHINEX, CMS …) • Grid Physics Network (GriPhyN) project funded by the Information Technology Research program of NSF ($10.9M for five years $2M to our group) • Joint UW-HEP and Condor project funded by the UW Graduate School. • 1M hours of Condor CPUs were allocated by the National Resource Allocation Board (NREC) to the CMS group at Caltech • Our team selected as the main CS group for the next phase of PPDG (DOE-SciDAC proposal pending) • CMS group of UW is one of the four sites of the Grid Laboratory of Wisconsin (GLOW) (NSF-MRI proposal pending)

  6. Physicists Have aLOTof data. (others have data problems too …)

  7. The Data Grids & Us • Extend and enhance existing capabilities of Condor • Develop new storage and data transfer management technologies (key element in PPDG proposal) • Interface “legacy” applications and the Grid “fabric” with new capabilities/technologies • Work closely with the Globus team and follow their leadership on data transfer and replica management tools (joint Data Grid SciDAC proposal is pending) • Close collaboration with the EU Data Grid effort • Make ClassAds the “Esperanto” of Grid management (joint Security SciDAC proposal with Globus is pending)

  8. And what about … • Allocating Network Resources, • Scheduling of Storage Resources, • Co-allocation of Processing, Storage and Network Resources, • Resource Reservation, • Data Shipping vs. Function Shipping, • Data Security, • Write Behind, • Data Affinity, • Or … ?

  9. Challenges Ahead • Ride the “Grid Wave” without losing our balance • Leverage the talent and expertise of our new faculty (Distributed I/O, Distributed Scheduling, Networking, Security) • Expend our UW-Flock to a state-wide system (WiscNet?) • Apply the Master Worker paradigm to domain decomposition problems (ongoing work with JPL) • Scale our Master Worker framework to 10,000 workers. • Open Source vs. Public Domain binaries vs. a Commercial version of Condor

  10. C High Throughput Computing ondor Don’t ask “what can the Grid do for me?”ask “what can I do with the Grid?”

More Related