Supercomputing project in CAS and its applications in earth sciences

1. Supercomputing project in CAS and its applications in earth sciences Xuebin Chi, Zhonghua Lu, Shaopeng Liu, Guihua Shan Supercomputing Center Computer Network Information Center (CNIC) Chinese Academy of Sciences (CAS)

2. Outline • Glance at Supercomputing Center of CAS • Computing Facilities • Software Installed • User Environment • Applications • Main Node of China National Grid(CNGrid) • Future Projects in Geosciences

3. Glance at Supercomputing Center • 5 divisions: System maintenance, Parallel computing research, Application software development, Grid technology application, and Scientific Visualization • Focus on parallel computing, parallel libraries, and provide solutions for complicated large scale applications in sciences and engineering • Objective: Provide computing and storage facilities and services

4. Computing Facilities • ICT Dawning 2000 • Peak: 111 Gflops • Memory: 46GB • Storage: 628GB • nodes: 82 • CPUs: 164 • double 333Mhz Power PC 604e (Thin/thick nodes); • double 200Mhz Power3 (High performance nodes, Main server nodes) • OS: AIX • Installation: May, 2000 • It is mainly for students to learn and practise HPC techniques freely.

5. Computing Facilities (con.) • Lenovo DeepComp 6800 • Peak: 5.3TeraFLOPS • HPL: 4.2 TeraFLOPS • Rank in TOP500: 14 (2003) • Number of nodes: 265 • Number of processors: 1060 • Processor: Itanium 2, 1.3 GHz • Memory: 2.6 TBytes • Storage: 80 TBytes • Network: Quadrics QSnet • OS: Redhat AS 2.1 • Installation :December, 2003

6. Computing Facilities (con.) • SGI Onyx 350 • Peak: 38GFLOPS • HPL: 22 GFLOPS • Number of processors: 32 • Processor: r16000, 600MHz • Memory: 32 GBytes • Storage: 500 GBytes • OS: IRIX 6.5 • Installation:March, 2004

7. Software Installed • Commercial Software • Gaussian 2003 • ADF • Molpro • VASP • LS-Dyna • Platform LSF • Totalview • Parawise

8. Software Installed (con.) • Free Software • LAPACK, ScaLAPACK • PETSc, FFTW • GAMIT • Paraview, VTK, Vis5D, Pymol, Rasmol, VMD, GMT • GAMESS, CPMD, DOCK,AUTODOCK, TINKER, GROMACS, NAMD2.5,Polyrate, almd       • Abinit, PDIP（dicvol,ito,treor） • BLAST, Clustalw, Blat, Predpharp • ……

9. Software Installed (con.) • Development of Application Software • Auto-Program Generator of FEM • Earth sciences( Active tectonic evolution, Crustal deformation , Mantle convection, Seismic Tomography Toolkits, LURR) • Bioinformatics(MPI_AltSplice, MPI_SiClone) • Computational Chemistry ( BDF, xian_CI, CCSD, SCIDVD, 3D ) • Material Science (Getstructure, MPI_PSSP) • Fluid Dynamics (OpenCFD) • …

10. User Environment • Scientific Computing Gird (ScGrid) Portal • User Module: Login & logout, User configuration, Job submission (batch and interactive), Job viewing, resource querying, file manager, file transfer, etc. • Accounting Module • Admin Module: Portal Administrator and CA manager • ssh, sftp

11. User Environment (con.)

12. User Environment (con.)

13. Applications • convection problem of mantle • Earthquake Prediction by LURR • Parallel finite element program generating system • New advances in 3-D seismic tomography • Carries out the Earth gravity field simulation • Climate Numerical Simulation • Research of Eco-Environmental Sciences • Research on the space weather

14. Results from this program 1、The remarkable success of the numerical simulation for convection problem of mantle • SCCAS supported the group of Prof.Yaolin Shi and Huai Zhang’s in Laboratory of Computational Geodynamics of the Graduate School of the Chinese Academy of Sciences to develop a large scale numerical simulation perform for computation convectional model of deficiently mixed mantle. • The large scale parallel computing finite element source program for simulating convection problem of mantle has been successfully used to simulate the Active Tectonic problem of Missouri-Columbia University of America. At the same time, the finite element software for simulating the coupling problem of 3-D temperature field and N-S fluid is being compiled and debuged, and will be completed soon.

15. 2、Earthquake Prediction by Applying Load/Unload Response Ratio (LURR) • SCCAS supported the group of Prof. Xiang Chu Yin,who work in the Institute of Earthquake Science, China Earthquake Administration and the Institute of Mechanics, Chinese Academy of Sciences to do the research of Earthquake Prediction, and got exhilarating results by applying LURR on the DeepComp 6800 supercomputer. LURR prediction software optimized by the SCCAS has been transplanted to the clusters of China Earthquake Administration. • The figure is the distribution map which shows the anomaly LURR regions in Chinese mainland in 2003, which was issued in the book “The research on seismic tendency of China in 2004”, edited by the Center for Analysis and Prediction, CSB, Seismological Press, Beijing, 2003, pp282-285

16. 2003.10.1-2004.9.30,0.125/0.125,0.0-4.0,y/yc,R=200 2004.4.1-2005.3.31, 0.25/0.25,0.0-4.0,R=300,15,90% Studied the earthquake tendency of 2005 in the end of 2004 • In 2004, 15 strong earthquakes with Ms≥5 among 17 (88%) fallen in to the LURR anomaly regions predicted in the end of 2003. • In the end of 2004 they conducted LURR spatial scanning of Chinese mainland to study the earthquake tendency of 2005. In 2005, 12 strong earthquakes with M≥5 among 13 fallen into the LURR anomaly regions. The Jiujiang earthquake with magnitude 5.7(November 26, 2005 ) is not included in the LURR anomaly regions in the figure. But it was reported on the Colloquium of earthquake tendency (March 31, 2005). 05年中图

17. Earthquakes prediction outside China • Up to now, 5 foreign earthquakes have been predicted in terms of LURR, among which 4 predictions were successful. • The south Asian earthquake (October 8, 2005, M7.6) • The Afghanistan earthquake (April 6, 2004, M6.8). • The seismic trend of Euro-Asian seismic belt began to be studied in the end of 2003. • The relative outcomes were issued on the Journal “Supercomputing Newsletter”, sponsored by the SCCAS.

18. An application of PFEG in the research on earthquake of Cascadia subduction zone in west of Canada (300 years later) 3、 Parallel finite element program generating system was developed to enhance scientific and engineering applications • Cooperated with the Prof. Guoping Liang’s of AMSS of CAS to develop the platform for generating parallel finite element program automatically by inputting PDE expressions and algorithms written by Liang’s FE language.. • The FE language presented by the platform is available for any physical filed, i.e. any PDE and finite element methods can be described without any difficulty. • Many remarkable results have been achieved.

19. 4、New advances in 3-D seismic tomography and application software development • SCCAS supported the group of Prof. F.T. Liu in the Institute of Geology and Geophysics to conduct the study of 3-D seismic tomography. • The large scale seismic tomography simulation validated a conclusion that the ultra high pressure rocks in China Dabie area are mainland plate which dive into mantle depth, and after the broken of the Ocean plate which drag it down, then returned to surface. Seismic tomography reveals that mainland deep subduction occured in Dabie-sulu area.

20. 5、Carries out the Earth gravity field simulation system research • SCCAS supported the team of Prof. Lu Yang from Institute of Geodesy and Geophysics of CAS carried out the Earth gravity field simulation system research. • In 2005, in the Mount Everest height determination work, the system, IGG05B (to 360 degrees), play a vital role in guaranteeing the accuracy of Mount Everest height result .

21. 6、Climate Numerical Simulation------《 IPCC Assessment Reports 》

22. 北京市地表覆盖现状 北京市城市热岛分布图 7、Research of Eco-Environmental Sciences • Cooperate with Prof. Ouyang Zhiyun (RCEES-CAS) and Dr. Liu Hongli (CAMS),Using Supercomputing for the research of eco-environmental sciences. This work includes heat island of city and how development of city influences eco-environment, which is useful for city planning. • Cooperate with Prof. Zhou Chunhong(CAWAS-CMA), developing a real-time numerical operational sand storm forecast system(CAWSDS-MM5). This system will be used for Asia sand storm forecast after middle March.

23. 8、Research on the space weather for China space projects • SCCAS supported the group of Prof. Chi Wang and Zhao-Hui Huang of CSSAR, CAS constructs the coupling model and synthetic forecasting method of adverse space weather, serves “Geospace Double Star Program” proposed first by theChinese scientists and investigates how interplanetary disturbances affect geospace. • Furthermore, their lab. is active proponents and participators of the international cooperative plan “Galileo” proposed by European Commission and Space Administration and China national major scientific projects “Meridian Project”.

24. 9、Developed a Remote Visualization system in high performance computing environment, providing an effective and convenient tool for scientists to analyze data • A basic visualization environment on the visualization server in Supercomputer center is built. And a grid-enable remote visualization service architecture is designed and implementation.； • The molecule visualization software “Gridmol” and LURR data visualization software “LURR_View” are developed； • ParaView, GrADs, GMT, Vis5D, GridMol, LURR_VIEW are all integrated in the remote visualization system, where a web-based friendly interface is provided to access all these visualization resources in supercomputing center. This offers an effective and convenient way for scientists to analyze data. VISGRID主页面 加卸载响应比数据可视化

25. Future Projects in Geosciences • New Supercomputer with 100Tflops partly supports Geosciences research • Virtual Lab for Geosciences • Parallel simulation of local strong earthquake, test-bed and applications • Geological disaster prediction based on LURR • Three dimensional tomography of nature earthquakes and seismological data processing • Large-scale simulation of seismology wave propagation and calculation of 3-D pre-stack depth migration

26. Thank you very much for your attention