Scalable Molecular Dynamics

1. Scalable Molecular Dynamics T.P.Straatsma Laboratory Fellow and Associate Division Director Computational Biology and Bioinformatics Computational Sciences and Mathematics Division Pacific Northwest National Laboratory

2. NWCHEM Integral API ENERGY Classical Force Field QMD DFT Geometry GRADIENT QM/MM SCF: RHF UHF ROHF Basis Sets OPTIMIZE ET MP2: RHF UHF PEigS QHOP DYNAMICS MP3: RHF UHF pFFT THERMODYNAMICS MP4: RHF UHF LAPACK INPUT ESP RI-MP2 BLAS PROPERTY VIB CCSD(T): RHF MA PREPARE CASSCF/GVB Global Arrays ANALYZE MCSCF ecce MR-CI-PT ChemIO CI: Columbus Full Selected NWChem Molecular Science Software 2

3. Two-dimensional representation Distributed data reduces memory use Locality of interactions reduces communication Fluctuating number of atoms requires atom redistribution Inhomogeneous distribution requires dynamic load balancing local node non-local node domain within short range non-local node domain within long range non-local node domain outside interaction range Domain Decomposition 3

4. coordinates x 1 1. Asynchronous ga_get of coordinates in box on neighboring node 2. Calculation forces with all local boxes within cutoff radius 3. Accumulate local forces 4. Asynchronous ga_acc to accumulate forces in box on neighboring node 2 F= -U(x) forces f 4 3 node j node i All data transfer by means of one-sided, asynchronous communication Force Evaluation 4

5. 1 2 3 4 5 6 7 All nodes Node sub-set Particle-mesh Ewald 1. Charge grid construction 2. Block to slab decomposition 3. 3D-fast Fourier transform 4. Reciprocal space energy & forces 5. 3D-fast Fourier transform 6. Slab to block decomposition 7. Atomic forces 5

6. Flowchart load balancing redistribution PME charge grid FFT & PME k-space synchronous communication record trajectory asynchronous communication get coordinates processor-sub set communication pair lists no processor communication forces global wait for processor sub set accumulate forces PME forces time step properties record properties 6

7. Synchronization PMEforces Non-local forces PME wait Local forces Reciprocal PME (fft, f-grid) PME node subset synchronization PME charge grid construction Timing Analysis Haloalkane dehalogenase, force evaluation timings 7

8. Local Redistribution Load Balancing Collective Resizing 8

9. Dynamic Load Balancing 9

10. Challenges for the DOE • Environmental Legacy at Hanford and other DOE sites • Bioremediation • Environmental and Health Impact of Energy Use • Carbon sequestration • Nitrogen fixation • Production of Energy • Biofuels • Hydrogen 10

11. Molecular Basis for Microbial Adhesion and Geochemical Surface Reactions Microbes in the subsurface mediate a number of environmental, geochemical processes: • Uptake of metal ions, including environmentally recalcitrant metals • Adhesion to mineral surfaces • Reduction and mineralization of ions at the microbial surface Pseudomonas aeruginosa: Cu, Fe, Au, La, Eu, U, Yb, Al, Ca, Na, K Shewanella putrefaciens: Fe, S, Mn Shewanella alga: Fe, Cr, Co, Mn, U Shewanella amazonensis: Fe, Mn, S Shewanella oneidensis MR1 External reduction involving OM cytochromes 11

12. Project Objectives Molecular level characterization of: • Microbial adhesion to mineral surfaces • Metal ion concentration in microbial membranes Focus on Gram-negative bacterial Outer Membrane Computational Approach: • Molecular modeling and molecular dynamics simulations • Quantum mechanical description of key functional groups • Thermodynamic Modeling 12

13. Gram Negative Cell Walls 13

14. MAN MAN O chain FUC 30-50 RHA GLC1 GLC3 GLC* RHA L-ALA GAL GLC2 Core LPS HEP2 CONH2 P HEP1 P P KDO1 KDO2 P NAG1 NAG2 P Lipid A LPS of Pseudomonas aeruginosa 1. Design of the Rough LPS Molecular Model 2. Determination of Electrostatic Model 14

15. LPS Membrane Construction Distribution of functional groups and water in the outer membrane of P. aeruginosa. These results are used for thermodynamic modeling of ion adsorption in microbial membranes. 15

16. Phosphate Clustering Outer Core Inner Core These results lend support to the interpretation of recent XAS experiments carried out by J. Bargar at SLAC indicating that uranyl ions take up by microbial membranes exists in clusters involving phosphates. 16

17. Membrane Electrostatic Potential Average Potential Across Membrane Calc.: 100 mV Exp.: 80 mV 17

18. Slab: Periodic Hartree Fock Fragment: Point Charges Blue: 25 e·kJ/mol Red: -25 e·kJ/mol Atomic Charges from 2D SCF-HF ESP Fit 18

19. 4 5 1 2 3 Membrane-Mineral Interactions 19

20. P. Aeruginosa Outer Membrane Proteins E. coli membrane protein TolC (Pautsch and Schultz, 1998) and homology modeled P. aeruginosa membrane protein OprM (Wong et al., 2001) E. coli membrane protein OmpA (Pautsch and Schultz, 1998) and homology modeled P. aeruginosa membrane protein OprF (Brinkman et al., 2000) E. coli membrane protein FecA (Pautsch and Schultz, 1998) and homology modeled P. aeruginosa membrane protein FecA (Straatsma, unpublished) 20

21. P. aeruginosa OprF 21

22. Electron transfer in bacterial respiration • Under anaerobic conditions, Shewanella frigidimarina is able to use extra-cellular iron as the electron acceptor in its respiration. The electron transfer pathway involves a number of cytochromes which deliver electrons from the cytoplasmic membrane to the periplasmic membrane, where iron reduction occurs. • The electron transfer (ET) between the membranes is carried out by the respiratory enzyme flavocytochrome c3 fumarate reductase (Fcc3), which contains four bis(histidine) hemes. 22

23. Electron Transfer in Fcc3 and Ifc3 Flavocytochrome c3 fumarate reductase of Shewanella frigidimarina 23

24. electronic coupling relaxation energy activation energy Low-spin electron transfer dz2 dx2-y2 dxz, dyz dxy Fe(III)2A2 Fe(II)1A1 3 2 4 1 Marcus’ theory of electron transfer 24

25. Heme-802 Heme-801 Heme-804 Heme-803 Fe(III) Ehs/ls Dels Dehs Ehs/ls Energy kcal/mol Fe(II) AEAls AEAhs Ehs/ls Dels Dehs Ehs/ls r(Fe-N) Å B3LYP Characterization of a model heme ET donor/acceptor orbital dπ 25

26. Computational Structural Biology Challenges • Computational protein structure prediction • Protein-protein complexes: cell signaling • Protein-membrane and mineral-membrane complexes • Extension to microsecond simulation times • Statistically accurate thermodynamic properties • Comparative trajectory analysis • Enzyme catalysis using hybrid QM/MM methods • Extension toward millisecond simulation times • Protein folding and unfolding • Membrane transport of simple ions and small molecules • Membrane fusion, vesicle formation • Scalability on next generation MPP and hybrid architectures 26

27. Acknowledgements Dr. Roberto D. Lins, ETH Lausanne, CH Dr. Robert M. Shroll, Spectral Sciences, Boston, MA Dr. Wlodek K. Apostoluk, Wroclaw University, Poland Dr. Andy R. Felmy, Chemical Sciences Division, PNNL Dr. Kevin M. Rosso, Chemical Sciences Division, PNNL Professor David A. Dixon, University of Alabama Dr. Erich R. Vorpagel, EMSL DOE Office of Advanced Scientific Computing Research DOE Office of Basic Energy Science, Geosciences Research Program DOE Office of Biological and Environmental Research EMSL Molecular Sciences Computing Facility Computational Grand Challenge Application Projects 27