Chapter 9 Cloud-Enabled Climate@Home

Chapter 9 Cloud-Enabled Climate@Home Jing Li, Zhenlong Li, Min Sun, Qunying Huang, Kai Liu, and Myra Bambacus

Learning Objectives • Climate@Home: background and challenges • Deploy the Climate@Home system onto the cloud • Run the system onto the cloud • Performance and cost analysis

Learning Materials • Online Videos: • Chapter_9_Video_1.wmv • Chapter_9_Video_2.wmv • Chapter_9_Video_3.wmv • Chapter_9_Video_4.wmv • Chapter_9_Video_5.wmv • Chapter_9_Video_6.mp4 • Chapter_9_Video_7.mp4 • Chapter_9_Video_8.mp4 • Scripts, Files and others: • Chapter_9_Section9.2.1.1.zip • Chapter_9_Section9.2.1.2.zip

Learning Modules • Climate@Home: background and challenges • Climate@Home Portal cloud deployment • Climate@Home Boinc cloud deployment • System demonstration • 5. Conclusion and discussions

Climate@Home: Background Climate modeling Data management and analysis Citizen computing

Climate@Home: the model • Model E (Schmidt et al., 2006): NASA Goddard Institute for Space Studies (GISS) • Coupled atmosphere-ocean models • Long term • Global coverage • Multidimensional • Monthly output • Chosen as the climate model for the Climate@Home project • URL: http://www.giss.nasa.gov/tools/modelE/

Climate@Home: the components • Spatial web portal • integrate all front-end components with back-end functions • geovisual analysis of the climate data and visual management of contributed computing resources (Sun et al., 2012) • access these components in a secured manner • The BOINC server • dispatch ModelE simulation tasks • collect model outputs • coordinat with the BOINC clients

Climate@Home: challenges

Cloud deployment: spatial web portal 1. Authorize network access 11. Build a new AMI from the running instance 2. Launch an instance 10. Configure the load balance/scalability 3. Create an EBS volume 9. Start the Apache2 and Tomcat server and configure the web portal 4. Attach the EBS volume to the instance 8. Restore the database 5. Install MySQL, Tomcat and Apache2 software package Steps 3,4,6,10 are optional, which are demonstrated in Chapter 8. 6. Mount the EBS Volume to the Data/Log directory of MySQL 7. Transfer the code/data of the web portal and visualization portlet

Step 1 and Step 2 1. Authorize network access 11. Build a new AMI from the running instance • Step 1. Authorize network access-- Three ports should be opened: Port 22, Port 80, and Port 8080. • Step 2. Launch an instance—A public AMI with Ubuntu 12.04 as OS is selected as the base image to create an instance for further customization. • After launching the instance, log on to the instance from a Windows machine using putty. 2. Launch an instance 10. Configure the load balance/scalability 3. Create an EBS volume 9. Start the Apache2 and Tomcat server andconfigure the web portal 4. Attach the EBS volume to the instance 8. Restore the database 5. Install MySQL, Tomcat and Apache2 software package • Video: Chapter_9_Video_1.wmv 6. Mount the EBS Volume to the Data/Log directory of MySQL 7. Transfer the code/data of the web portal and visualization portlet • Files/Tools used in this deployment: Chapter_9_Section9.2.1.1.zip

Step 3, 4 and 6 1. Authorize network access 11. Build a new AMI from the running instance • Steps 3, 4, and 6. Configure EBS—Steps 3, 4, and 6 are not described in this chapter. The detailed configurations have been discussed in Chapter 8, Section 8.2.2.1. 2. Launch an instance 10. Configure the load balance/scalability 3. Create an EBS volume 9. Start the Apache2 and Tomcat server andconfigure the web portal 4. Attach the EBS volume to the instance 8. Restore the database 5. Install MySQL, Tomcat and Apache2 software package 6. Mount the EBS Volume to the Data/Log directory of MySQL 7. Transfer the code/data of the web portal and visualization portlet

Step 5Install software packages 1. Authorize network access 11. Build a new AMI from the running instance • In this step, Apache2 server, Tomcat server, and MySQL are installed and configured as required by the spatial Web portal. 2. Launch an instance 10. Configure the load balance/scalability 3. Create an EBS volume 9. Start the Apache2 and Tomcat server andconfigure the web portal 4. Attach the EBS volume to the instance 8. Restore the database 5. Install MySQL, Tomcat and Apache2 software package • Video: Chapter_9_Video_2.wmv 6. Mount the EBS Volume to the Data/Log directory of MySQL 7. Transfer the code/data of the web portal and visualization portlet

Step 7Transfer files through WinSCP 1. Authorize network access 11. Build a new AMI from the running instance • Drupal source codes are moved to the web folder of Apache HTTP server. • The geovisual analytical portlets are moved to the web folder of the Tomcat server. • Database files should be moved to a secured folder in the instance for later restoration. 2. Launch an instance 10. Configure the load balance/scalability 3. Create an EBS volume 9. Start the Apache2 and Tomcat server andconfigure the web portal 4. Attach the EBS volume to the instance 8. Restore the database 5. Install MySQL, Tomcat and Apache2 software package • Video: Chapter_9_Video_3.wmv 6. Mount the EBS Volume to the Data/Log directory of MySQL 7. Transfer the code/data of the web portal and visualization portlet

Step 8. Restore the database 1. Authorize network access 11. Build a new AMI from the running instance 2. Launch an instance The Portal database and Boinc server database are restored in this step. 10. Configure the load balance/scalability • Video: Chapter_9_Video_4.wmv 3. Create an EBS volume root@ip-10-189-149-104:/mnt$ mysql –u username –p #log in the mysql database using root mysql>create user ‘climateuser’@localhost identified by ‘password’ #create a user account (e.g. climateuser) for the portal mysql>create database climatehome#the name of the climate@home drupal database mysql> grant all privileges on climatehome .* to ‘climateuser’@localhost identified by ‘password’ #grant permission to the database mysql>flush privileges mysql>exit #quit mysql root@ip-10-189-149-104:/mnt$ mysql -u username -p climatehome< climatehome.sql #import the climatehome database climatehome.sql on the cloud server 9. Start the Apache2 and Tomcat server andconfigure the web portal 4. Attach the EBS volume to the instance 8. Restore the database 5. Install MySQL, Tomcat and Apache2 software package 6. Mount the EBS Volume to the Data/Log directory of MySQL 7. Transfer the code/data of the web portal and visualization portlet Command to restore a MySQL database

Step 9. Configure the spatial web portal and visualization portlet 1. Authorize network access 11. Build a new AMI from the running instance 2. Launch an instance • Start the Apache2 and Tomcat server and configure the web portal. The following commands can be used to start the servers on the Ubutun instance. • root@ip-10-189-149-104:/$ sudo service apache2 start • root@ip-10-189-149-104:/Tomcat/bin/$ ./startup.sh 10. Configure the load balance/scalability 3. Create an EBS volume 9. Start the Apache2 and Tomcat server andconfigure the web portal 4. Attach the EBS volume to the instance • Once the two servers are started, the web portal can be accessed from http://Your_VM_Public_DNS(IP)/climate@home/. Please refer to the video for the detailed steps. • Portal login information: • User name: admin • Password: cisc2345!@#$GD 8. Restore the database 5. Install MySQL, Tomcat and Apache2 software package • Video: Chapter_9_Video_5.wmv 6. Mount the EBS Volume to the Data/Log directory of MySQL 7. Transfer the code/data of the web portal and visualization portlet

Step 10 and 11 1. Authorize network access 11. Build a new AMI from the running instance • Step 10 is an optional step to make the system more flexible and scalable. The configuration is introduced in Section 8.2.2.2 and 8.2.2.3. • The last step is to create a new AMI based on the running instance of the Climate@Home portal. Chapter 4, Section 4.3.1 describes the creation of an AMI. The image can be reused to create multiple virtual instances. 2. Launch an instance 10. Configure the load balance/scalability 3. Create an EBS volume 9. Start the Apache2 and Tomcat server andconfigure the web portal 4. Attach the EBS volume to the instance 8. Restore the database 5. Install MySQL, Tomcat and Apache2 software package 6. Mount the EBS Volume to the Data/Log directory of MySQL 7. Transfer the code/data of the web portal and visualization portlet

Cloud deployment: the BOINC server

Step 1 and 2Authorize network access andlaunch an instance • Step 1. Authorize network access—Two ports should be opened: Port 22, Port 80. Port 22 is used to access the instance through the Secure Shell (SSH). Port 80 is for HTTP. • Step 2. Launch an instance—A public AMI with Ubuntu 12.04 as the OS is selected as the base image for creating an instance for further customization. • Video: Chapter_9_Video_6.mp4 • Files/Tools used in this deployment: Chapter_9_Section9.2.1.2.zip

Step 3, 4 and 6 • The Climate@Home Boinc server uses distributed FTPS server to store the uploaded results. Hence, EBS is optional for the Boinc server. The detailed configurations have been discussed in Chapter 8, Section 8.2.2.1.

Step 5 and 7Install software packages and Boinc server • Step 5. Install software packages—Several software packages are needed for the BOINC to function, including MySQL, Apache HTTP server, Python, PHP5, and others. • Step 7. Install BOINC server software. • Video: Chapter_9_Video_6.mp4

Step 8 Create a BOINC project and configure BOINC server Upon finishing the installation of the BOINC server, a BOINC test project should be created, which distributes the ModelE applications to computing devices contributed by citizens. • Video: Chapter_9_Video_7.mp4

Step 9,10 and 11 • Step 9. Upload the ModelE VirtualBox image and ModelE configuration files to the instance. The ModelE VirtualBox Image is put into the application folders of the BOINC server. The ModelE configuration files are put into the download folder of the “climateathome” project created in Step 8. • Step 10. Configure the load balance or scalability—This is an optional step. The configuration is introduced in Section 8.2.2.2 and Section 8.2.2.3 • Step 11. Create work units and start the project—Consumers need to create input and output templates to start the project. • Video: Chapter_9_Video_8.mp4

Step 12Create an AMI from the running instance • This step creates a new AMI based on the running instance of the BOINC server for reuses. Please see Chapter 4, Section 4.3 for creating an AMI.

The home page of the Climate@Home spatial web portal

Seasonal mean analysis using model outputs

Visualizing volunteered computing nodes with Google Map (Li et al., 2013)

Cloud services in the Climate@Home system

Conclusions • Deploying the spatial web portal to the cloud service handles concurrent access to the online geovisual analytical portlet • Deploying the BONIC server in a cloud environment ensures the robustness, high performance, security and reliability of managing model simulations.

Discussion Questions Using the Climate@Home system as an example, list the major system components required for supporting climate modeling and analysis in a distributed environment. Using the development of Climate@Home system as an example, explain why cloud computing techniques are essential in supporting climate modeling and analysis in a distributed environment. What are the major steps of deploying the spatial web portal onto the cloud services? In the process of deploying the spatial web portal, what are the special considerations related to network configurations? What are the major steps of deploying the BONIC server onto the cloud services? What are the special considerations compared to the general application deployment in the Chapter 5? What are the hardware configuration requirements of a virtual machine serving as a BONIC server? What are the usages of an Elastic Block Store (EBS) in the cloud-enabled Climate@Home system? List the cloud services used in the deployment of the Climate@Home system. What are the remaining issues for deploying the Climate@Home components onto the cloud?

References • Li, Z., C. Yang, M. Sun et al. 2013. A High Performance Web-Based System for Analyzing and Visualizing Spatiotemporal Data for Climate Studies. In Web and Wireless Geographical Information Systems, pp. 190-198. Springer Berlin Heidelberg, 2013. • Sun, M., J. Li, C. Yang et al. 2012. A Web-Based Geovisual Analytical System for Climate Studies. Future Internet 4, no. 4 : 1069-1085. • Schmidt, G.A., Ruedy, R., Hansen, J.E. et al. 2006. Present-Day Atmospheric Simulations Using GISS ModelE: Comparison to In Situ, Satellite, and Reanalysis Data. Journal of Climate 19, 153–192.

Chapter 9 Cloud-Enabled Climate@Home