School of Earth Science computational research

1. School of Earth Science computational research

2. Outline • SES research from a computing perspective • SEES enabled research • Modeling • Sparse matrix inversion • Future hardware technologies, geoscience computing, and CEES CESS advisory board 2006

3. Types of CEES users: Running canned software Canned • Run canned software • Eclipse • Promax • No programming skills or need for programming in their research • Run on our shared memory machine • CEES allows them to run larger models, do more simulations CESS advisory board 2006

4. Types of CEES users: Interpreted programming Canned • Run matlab, femlab, etc • Minimal programming skills • Run on our small shared memory machines • CEES allows them to run larger models, do more simulations Interpreted CESS advisory board 2006

5. Types of CEES users: Serial programmers Canned • Write their own code or build upon someone else’s existing code • Run on our small shared memory machine • Larger problem size, more simulations Interpreted Serial CESS advisory board 2006

6. Types of CEES users: OpenMP programmers Canned • Write their own code or modify others shared memory, OpenMP code • Run on both our small and large shared memory machines • We enable larger problem sizes, more simulations Interpreted Serial OpenMP CESS advisory board 2006

7. Types of CEES users: MPI users Canned • Write or build upon Message Passing Interface based code • Run on cluster • Enable larger problem size, more simulations • Also enable a new class problems to be addressed because of the low latency, high bandwidth network Interpreted Serial OpenMP MPI CESS advisory board 2006

8. Types of CEES users: MPI users Canned • Write or build upon Message Passing Interface based code • Run on cluster • Enable larger problem size, more simulations • Also enable a new class problems to be addressed because of the low latency, high bandwidth network Interpreted Serial OpenMP MPI CESS advisory board 2006

9. SES research common themes • Modeling earth processes (regular or irregular grid) • Finite element • Domain decomposition • Sparse matrix inversions CESS advisory board 2006

10. Air-Sea CO2 Exchange in the Ross Sea, Antarctica The amount of atmospheric CO2 entering the surface ocean is proportional to the partial pressure of CO2 (pCO2) pCO2 in the upper ocean is under strong biological control Bio-oceanography Arrigo

11. Air-sea exchange is greatest when pCO2 is low pCO2 (µatm) 450 400 350 300 250 200 150 100 50 0 Dark blue = sea ice covered

12. Bio-oceanography • Currently a serial application • Loosely coupled domain • Good candidate for parallel implementation

13. Simulation Work Flow Earth Model Well Logs Seismic Interpretation Mapping Gridding Reservoir Simulation Stratigraphic Modeling CPU 4 CPU 3 CPU 2 CPU 1 Scale-Up Well Planning Surface Facilities Rock & Fluid Analysis Production Data Reservoir flow simulation Tchelepi

14. Complex Grids Flexible grids have advantages but difficult to use Automatic grid generation! Gurpinar, 2001 Prevost, 2003 Wolfsteiner et al., 2002 CESS advisory board 2006 Reservoir flow simulation Tchelepi

15. General Purpose Reservoir Simulator (GPRS) • Tightly coupled, finite element modeling • Involves solving a large sparse matrix • Currently being transformed from a serial to OpenMP program CESS advisory board 2006

16. Unsolved Mystery Monterey Bay Facts • Large internal tides • Swift currents • Elevated dissipation • Steep / rapidly changing bathymetry How do internal tides evolve and are they responsible for the unexplained elevated dissipation? Effects on: oil rigs, pipelines, riser pipes, acoustics, ocean food web, sediment transport Computational fluid dynamics Gerritsen

17. River Flow Professor Margot Gerritsen • Petroleum Engineering CESS advisory board 2006 Computational fluid dynamics Gerritsen

18. Green: Water-filled pores Red: Oil White: Grains 10 mm Two-phase Pore Scale Flow (Lattice Boltzmann Simulation) Keehm, Stanford Geophysics CESS advisory board 2006 Computational rock physics Nur and Mavco

19. Fluid flow simulations • Tightly coupled systems • Finite element implementations • Current MPI implementations CESS advisory board 2006

20.       Modeling Study Model 0 * Depth (m) 64 m 60 50 m 15 50 Time (ms)

21.       Modeling Study Model Seismogram 0 * Depth (m) 64 m 60 50 m 15 50 Time (ms)

22.       Modeling Study Model Seismogram 0 * Depth (m) 64 m 60 50 m 15 50 Time (ms)

23.       Modeling Study Model Seismogram 0 * Depth (m) 64 m 60 50 m 15 50 Time (ms)

24. Field data P S TS TP    TST PT  ST TPT   Modeling Study Seismogram 0 * Depth (m) 64 m 60 50 m 15 50 Time (ms)

25. Field Data Modeling Blocked Vp log Model 8190 R      Depth (ft)  S 9200 640 (ft)

26. Seismograms Synthetic Field P S TS TP TS P S TP TST TPT TST PT PT ST ST TPT

27. CESS advisory board 2006

28. CESS advisory board 2006

29. Modeling similarities • Large computational domain that is to large for a single CPU • Computational requirements are to large for a single processor • Data domain can be sub-divided but require a high speed interconnect to avoid communication time dominating CESS advisory board 2006

30. Background • Shi and Malik (2000) developed the normalized cuts image segmentation for partitioning images. • Hale and Emanuel (2002, 2003) adapted the technique to painting 3D atomic meshes using coherency data.

31. Global segmentation for salt picking • Global solution y Depth x y

32. Segmentation methodology • Define weights relating each pixel to every other pixel in a local neighborhood. • The image is partitioned so that the normalized cut is minimized:

33. Minimize the normalized cut • Construct a weight matrix: W • Calculate a diagonal matrix: D • Set up the generalized eigensystem: • The second smallest eigenvector partitions the image so that the normalized cut is minimized.

34. Multiple attributes Amplitude • Amplitude alone is not always the best boundary indicator. • We combine multiple attributes to improve tracking capability. Dip variability and Amplitude

35. Combination picking result

36. Amplitude

37. Section 1

38. Section 2

39. Section 3

40. Section 4

41. Section 5

42. CEES research goals • Provide a computing resource for high level users • Upgrade SES computing skills so larger problems can be addressed • Provide unique resources that SES groups can’t provide • Influence hardware development for the benefit of geoscience applications CESS advisory board 2006

43. Types of CEES users • Canned software users • Interpreted languages • Serial programmers • OpenMP programmers • MPI Programmers CESS advisory board 2006

44. CEES users CESS advisory board 2006

45. EEES157 first day CESS advisory board 2006

46. EEES157 last day CESS advisory board 2006

47. Improving SES programming abilities • MPI is to large a jump for most users • The basics of OpenMP is easily understandable to most students • Efficient serial programming skills are not taught • Thinking about algorithmic improvements offers the most potential speedup CESS advisory board 2006

48. Future of computing • Clock speed improvements have stalled if not ended • Parallel operations are the short and mid-term solution (multi-core, hardware accelerators) • Compilers will not do the work for you in most cases • How programming is done is undergoing a fundamental change • Programming paradigms must change CESS advisory board 2006

49. Hardware accelerators • Moore’s law is ending/changing • Multi-core can only go so far • Floating point units becoming less important to chip makers • Web servers and many business applications are integer driven CESS advisory board 2006

50. Hardware accelerators • Game industry has turned to performing higher percentage of operations on hardware accelerators (Graphics processing units) and now the Cell processor • For certain tasks these accelerators are 5-20 times faster than today’s processors • Projected speed curves show the speed up increasing CESS advisory board 2006