EU Astrophysics Network Overview

1. EU Astrophysics Network Overview Ed Seidel Albert Einstein Institute Principal Network Coordinator www.eu-network.org Sources of Gravitational Radiation

2. Scientific and technological reasons for carrying out research in our field • Exploring Einstein’s General Relativity Want to develop theoretical lab to probe this fundamental theory • Fundamental theory of Physics (Gravity) • Among most complex equations of physics • Predict black holes, gravity waves, want much more • Exciting new field: Gravitational Wave Astronomy • LIGO, VIRGO, GEO, LISA, … ~ $1 Billion worldwide! • Fundamentally new information about Universe • A last major test of Einstein’s theory: do GWs exist? A century later, both of these developments happening at the same time: very exciting coincidence! • But, the community needed to carry out theoretical work is lacking… Not enough people or training www.eu-network.org Sources of Gravitational Radiation

3. Grand Challenge Collaboratories Building Communities to Solve These Problems • NASA Neutron Star Grand Challenge • 5 US Institutions, 3years, $1.4M • Solve problem of colliding neutron stars (try…) • NSF Black Hole Grand Challenge • 8 US Institutions, 5 years, $4M • Solve problem of colliding black holes (try…) • This EU Network Astrophysics • 10 EU Institutions, 3 years, €1.5M • Continue these problems • Entire Community becoming Grid enabled • NSF ASC Project • 5 US Institutions, 3 years, $2.2M • Develop “Collaboratory” • EU GridLab • 10 EU Institutions, 3 years, €5M • Develop Grid Tech • For these projects www.eu-network.org Sources of Gravitational Radiation

4. Actually came up with an answer to this one! We work hard together, but… We have fun, too. Actively building our Network community Probably first Network to support socio-economic tourism with new € Listening to a talk Explaining BH Spin Having Fun Socio-Economic Aspects of our Project for the EU… www.eu-network.org Sources of Gravitational Radiation

5. Network research objectives • The formation of a close alliance among the different expert groups to solve urgent problems required for GWA, too large and complex for any single group. • Such a Network was needed, but did not exist. • The development and training of a young community of researchers to populate this emerging research area of GWA. • Strong, but somewhat isolated European groups already in relevant areas • Need coordinated effort to build up the broad-based expertise required for a systematic attack on these problems. • The development of a community simulation code for relativistic astrophysics • Simulation tools are essential for the success of this research area. • Cactus is an important part of the training program for this Network. www.eu-network.org Sources of Gravitational Radiation

6. The development of appropriate approximation tools to provide better physical understanding and confirmation of numerical simulations. • Perturbation theory • Post-Newtonian Theory • The application of these numerical and approximation tools to a set of core astrophysics problems • GW signals from NS, BS, ??? • Most promising sources for LIGO/GEO/VIRGO, … • Good focus problems for training a community www.eu-network.org Sources of Gravitational Radiation

7. Computational Tools Theoretical Tools Einstein Eqs Post-Newtonian Theory Cactus Viz Pert. theory Grid KDI General View of Network Enablers Focus Physics Areas Other (strange stars, Core collapse, etc) 3D NS processes and mergers 3D BH Collisions World ZIB/Garching LBL/NCSA/ANL Outreach /Collab Cactus Community GridLab NCSA/WashU NSF KDI www.eu-network.org Sources of Gravitational Radiation

8. BH Data/Evolutions Post-Newtonian Schemes Characteristic Codes Perturbative Time Evolutions Cactus Dev/Training Nonlinear GR Hydro And Applications Stellar Pert Theory Ports Potsdam Ports Ports Potsdam Potsdam SOTON CS Efforts Worldwide SOTON SOTON Jena Jena Jena Jena Meudon Meudon Trieste Trieste AUTH Rome AUTH AUTH Rome Rome Valencia Valencia Valencia Valencia Valencia Palma Palma Palma Palma Our Team7 Focus areas, links created, strengthened by Network Ports Potsdam SOTON Jena Jena Meudon WashU Trieste AUTH Rome Valencia Palma www.eu-network.org Sources of Gravitational Radiation

9. Super EEs Super EEs EE’s Gauges MOL Parallel I/O Elliptics AMR Horizon Finders Wave Extractors Lazarus Project View: Code Leverage Vacuum BH Studies NS Studies Meudon BNS Projects extremely Collaborative Build on each other Share Modules or Build them together Cactus an important collaborative technology Meudon BBH Hydra EE’s Gauges MOL Parallel I/O Elliptics AMR EE’s Community Workshop this spring/fall improves EE’s, all simulations Horizon Finders Wave Extractors Lazarus www.eu-network.org Sources of Gravitational Radiation

10. Developing a Community:People Leverage • Actual Network funding rather small • 10-12 positions (some positions converted to 2 for shorter times) • 6-7 postdocs, 4-5 predocs • Very Strong Leverage! 6:1 ratio of paid/volunteer effort • 10 coordinators • 65 others officially working on the project at least part-time • Other sites in Europe joining in • Is it possible to get additional funding to help them? • Other projects in US and EU adding value • Attending this meeting • ~65!! • This project has seeded a much larger effort • Now • Even more later www.eu-network.org Sources of Gravitational Radiation

11. Community Code: Advantages • Sharing of Expertise: No single group or community can do these problems • Sharing of Code among projects • Infrastructure • BH routines apply directly to NS work, etc… • But free to keep routines in group as long as desired • Better Code • Open source encourages people to be more careful in coding! • Encourages documentation • Encourages deeper thinking about how it interfaces to another code • More trusted code, results • When code becomes open, and people can run it for themselves, they will begin to believe the results • Improvements propagate quickly though community • Not well accepted yet, but we are starting a good trend… www.eu-network.org Sources of Gravitational Radiation

12. Scientific Highlights of Work Thus Far • The Network is working • Community developing • Exciting meetings • Tools being used • Extraordinary new BH evolution results between AEI/Meudon groups • New Hydro Effort between AEI/SISSA/Thessaloniki/Valencia • … www.eu-network.org Sources of Gravitational Radiation

13. NSF Grand Challenge 1994 Equal mass, no spin, no J Head on Collision in 3D Code crashes by t=40M See New Diener Result!! Grazing Collision 1999 Big Advance: 3D, unequal mass, spin, J Physics! Code crash by 40M Pre-Network 3D BH Results www.eu-network.org Sources of Gravitational Radiation

14. New 3D Black Hole Evolutions • New Techniques • Excision • Gauges, formulations • Baumgarte initial data • Estimated Innermost Stable Circular Orbit (ISCO) • Other circular orbits, too (“Pre-ISCO sequence: PI-1-10) • OP Group has generated most astrophysically relevant binary BH data to date: now in Cactus! www.eu-network.org Sources of Gravitational Radiation

15. High ProfileThese simulations will break ground in various ways • We feel we are about to crack (not solve…) this problem • Stumped community for 4 decades • $4.5M NSF Grand Challenge project helped lay groundwork • No other group close to doing a BBH orbit • US Dept of Energy • Special allocation for us to do work on 4.5TB machine • Very unusual: DOE HQ approval given! • NCSA • Special allocation • Public Outreach • Discovery Channel Movie for June • Scientific American Cover in April www.eu-network.org Sources of Gravitational Radiation

16. EU Hydro Code EffortAEI/Valencia/Thessaloniki/Trieste • Originally a small collaboration between AEI and WashU • NASA GC project • Several papers published jointly, code developed/tested jointly • Hydro code itself primarily written by Mark Miller at WashU • Decided in Dec, 2001 to develop independent EU code, based on original lessons learned from old code • Visit of Baiotti and Montero to AEI, with Ian Hawke, many others contributing, critical to Network Hydro code • From start to finish, about 3 months • Initial Applications • Mergers: NS-NS, strange-strange, accretion • Later Applications • NS-BH, etc… • Stellar Core Collapse (w/Dimmelmeier) www.eu-network.org Sources of Gravitational Radiation

17. The methodological approach and work plan • Scientifically organized into Fully Numerical, Post-N, and Perturbation Theory • Management: Project and Institutional tracks • Each subarea has person clearly assigned to shepard activities • Facilitating Connections • Phone calls, conference calls, Emails, etc • Each Institute has Local Coordinators assigned to manage Institutional matters • Project Coordinator tries to have global view of things • Advisory Board gives feedback • Students feedback solicited, incorporated • Peter Diener assigned to be student Ombudsperson… www.eu-network.org Sources of Gravitational Radiation

18. Advisory Board • Bernard Schutz • AEI Director • Heavily involved in GEO, other GW Detector Efforts • Tsvi Piran, Hebrew University • Pioneer in numerical relativity • Very familiar with both relativity and astrophysics problems we face • Karsten Danzmann • AEI Director • Leads GEO Project • Jean-Pierre Lasota • Astrophysicist/Relativist • A Leader of Meudon group, expert in many problems of interest to us • Wai-Mo Suen • Close colleague, involved in many similar projects • Leader of some major US-based projects (ASC, NASA, others) www.eu-network.org Sources of Gravitational Radiation

19. My Favorite Advisory Board Member • “Sathya” • Expert in many things • Very polite • A true gentleman www.eu-network.org Sources of Gravitational Radiation

20. Work Plan Cactus Training complete, AMR tested Remote/distributed Simulations, input from projects below Full community Code release, with documentation Characteristic Methods Addressing Efficacy Of Characteristic Hydro Nonlinear Numerical Codes, Simulations Nonlinear BH simulations Merger simulations with studies of waves Continually refined as techniques developed Hydrodynamics Completed binary NS Initial Data Module Accretion Code Simulations refined as techniques developed Post-Newtonian theory PN GW, reaction expressions Post-N.Hydro Code, Post-N Initial Data Module Simulations continue Stellar perturbation theory Pert Eqs. for rotating NS, nonlinear results Modes computed, valuable input to num. simulations Pert. Techniques, codes, Perturbative time evolutions Close limit module, time evolution for rotating NSs Perturbative evolutions, Input to num. simulations 36 months 12 months 24 months Time www.eu-network.org Sources of Gravitational Radiation

21. How the network functions and how the partners collaborate • Our Website for communications: • http://www.eu-network.org/ • Notes • Codes • Workplans • Results • Newsletter • Visits • Exchanges • AEI Visitor Program • Conference calls • All Pis every 2 months • Various subgroups at regular intervals • Video Conferencing: (Apply for a supplement??) www.eu-network.org Sources of Gravitational Radiation

22. Electronic Collaborations • CVS for common source code, reports, etc • Central repository at AEI • All partners in Network (and worldwide) can update local installations via simple commands (cvs update) • Email • Cactus • Collaborative • Modular • Testsuites! • Very advanced computational tools, developed worldwide, used, tested by our group • Leveraging Technologies of ASC, TiKSL, GridLab projects www.eu-network.org Sources of Gravitational Radiation

23. Leveraging Other Projects • Cactus • NSF KDI-ASC Project • Suen (WashU), Norman, Shalf (LBL),Seidel (AEI/NCSA), Foster (Argonne), Parashar (Rutgers) • Norman’s Zeus Astrophysics Suite • GrACE: Parallel AMR for Many Applications • Portal: Global Resource Management, building generic GUI interfaces for resource selection • Viz Tools • German DFN TiKSL/GriKSL projects • GridLab www.eu-network.org Sources of Gravitational Radiation

24. Cactus Computational Toolkit Science, Autopilot, AMR, Petsc, HDF, MPI, GrACE, MDS, Remote Steering... The Astrophysical Simulation Collaboratory (ASC)$2.2M NSF project (Suen, Seidel, Shalf, Foster, Norman, Parashar,…) 1. User has science idea... 2. Composes/Builds Code Components w/Interface... 3. Selects Appropriate Resources... 4. Steers simulation, monitors performance... 5. Collaboratorslog in to monitor... Want to integrate and migrate this technology to the generic user… www.eu-network.org Sources of Gravitational Radiation

25. CactusOur collaborative Infrastructure remote steering Plug-In “Thorns” (modules) extensibleAPIs ANSI C Fortran/C/C++ parameters driver scheduling equations of state Core “Flesh” input/output errorhandling black holes interpolation makesystem boundaryconditions gridvariables SOR solver coordinates multigrid wave evolvers www.eu-network.org Sources of Gravitational Radiation

26. Cactus User CommunityUsing and Developing Physics Thorns Numerical Relativity Other Applications AEI Southampton Wash U RIKEN Chemical Engineering (U.Kansas) Goddard Penn State Thessaloniki Climate Modeling (NASA,+) Impact on other Projects! Tuebingen TAC SISSA Portsmouth EU Astrophysics Network NASA Neutron Star Grand Challenge Bio-Informatics (Canada) Geophysics (Stanford) Early Universe (LBL) Meeting Next Year: Theoretical Foundations of Einstein’s Equations for Numerical Relativity (Using Cactus to investigate and compare different implementations) Plasma (Princeton) Astrophysics (Zeus) www.eu-network.org Sources of Gravitational Radiation

27. Cactus ASC Portal www.ascportal.org (prototype/works now) • Part of NSF KDI Project (Astrophysics Simulation Collaboratory) • Use any Web Browser !! • Portal (will) provides: • Single access to all resources • Locate/build executables • Central/collaborative parameter files, thorn lists etc • Job submission/tracking • Access to new Grid Technologies www.eu-network.org Sources of Gravitational Radiation

28. GridLab: www.GridLab.orgEnabling Dynamic Grid Applications • EU Project ~€5M • AEI, ZIB, PSNC, Lecce, Athens, Cardiff, Amsterdam, SZTAKI, Brno, ISI, Argonne, Wisconsin, Sun, Compaq • Grid Application Toolkit (APIs/Tools) • Develop new grid scenarios for 2 main apps: Numerical relativity Grav wave data analysis www.eu-network.org Sources of Gravitational Radiation

29. SDSC S Brill Wave RZG SDSC LRZ S1 Calculate/Output Invariants S2 Archive data P1 Found a horizon, try out excision P2 Calculate/Output Grav. Waves Look for horizon S2 S1 Archive to LIGO public database Find best resources P2 P1 NCSA Dynamic Grid Computing Add more resources Queue time over, find new machine Free CPUs!! Clone job with steered parameter Physicist has new idea ! www.eu-network.org Sources of Gravitational Radiation

30. Connections to industry • We are connected to industry through computational science projects • Microsoft • Sun • Compaq • Intel • GridLab • Grid Systems • No direct coupling through our science (that I know of) www.eu-network.org Sources of Gravitational Radiation

31. The training programme • Feedback on talks, practice talks, etc • Review Talk topics selected by YR’s • School planned for 2003 • Training sessions (twice), extensive online tutorials • High performance computing, • Cactus (our code framework) • Visualization tools, • Code maintenance systems (CVS), etc • Want to give Video courses on scientific topics! • Not able so far • Access Grid funding with NSF? www.eu-network.org Sources of Gravitational Radiation

32. Use of the Budget • Midterm report shows basically on track • Most of the money (1.5M€) spent on YR • About 1/3 of funding spent to date • Roughly proportionate for Networking and salary costs • Will accelerate spending as more YRs hired • Portsmouth, Valencia, AUTH made late hires • UIB just now making hires • Any proposed revision to the contract? • No major changes forseen at this time, but questions www.eu-network.org Sources of Gravitational Radiation

33. Young Researcher Issues • Special attention paid to their training and well being… • Any difficulties encountered? Peter will report! • Responsibilities given to YRs • Peter Diener’s role as YR coordinator • Helping to prepare reports! • Developing Web sites, documentation • Building Central Codes • Developing “soft skills” • Collaboration/Communication built in to this project www.eu-network.org Sources of Gravitational Radiation

34. Seek Input • Actively seek YR input into all activities of the Network, • Owe several beers: Dorota, Johannes, Reinhard • 1.5 cases to Denis, Werner… • topics for review talks, suggestions for improving their training • Anonymous FAQ on web site • Give Input/Advice • Comment constructively on the talks • Last time had rule that senior researchers could not ask questions until after YRs did: wait and prod until they do! www.eu-network.org Sources of Gravitational Radiation

35. General Problems Encountered • Recruitment difficult • Restrictive rules • No existing EU community • Travel budgets not enough • Restrictions on overseas travel! • Insufficient EU travel/subsistence funding for visits • No real budget to support extended visits we are supposed to have • Bureaucracy • Reports, regulations far more than any NSF, NASA, or DFN grant I know! • Computing • No central EU computing facilities • We are quite lucky and unique due to US connections www.eu-network.org Sources of Gravitational Radiation

36. Specific Questions from/for EC • Staffing issues/Collaborations • Students finishing after Network ends • Must care for them! • No-Cost Extension? • New Project/Continuation • Status of UIB • Strong historical connection to AEI • Postdocs hired beginning next month and month after • Generally plan to fulfill person months • Could use some supplement for extended visits • Role of WashU group www.eu-network.org Sources of Gravitational Radiation

37. Strengthening Ties with Other Projects • US groups • WashU • NSF projects, comoputing facilities, etc • New NSF, NASA projects? Joint funding? • GridLab, other EU projects • GridLab officially connected through Pis, technologies • Enabling other EU groups to join as “Associate Members” • Various other groups want to join • A small amount of funding could go a long way • Various issues to discuss later… www.eu-network.org Sources of Gravitational Radiation