Developing an Agricultural Monitoring System from Remote Sensing Data Using GridRPC on Ninf-G

1. Developing an Agricultural Monitoring System from Remote Sensing Data Using GridRPC on Ninf-G Shamim Akther, Yann Chemin, Honda Kiyoshi Asian Institute of Technology, Thailand Itthichok Jangjaimon, Putchong Uthayopas Kasetsart University, Thailand And Thai National Grid Project

2. Motivation • Recent advances in data assimilation of satellite imagery for the purpose of agricultural monitoring led to the increasing demand for computing power • There is a need for technology and platform that can meet the requirement

3. SWAP Model and Structure Adopted from Van Dam et al. (1997) Drawn by Teerayut Horanont (AIT)

4. SWAP-GA • Know SWAP Input parameter – sowing date, soil property, Water management, etc • Finding SWAP Crop model

5. SWAP Model Parameter identification - Data Assimilation using RS and GA SWAP Input Parameters sowing date, soil property, Water management, and etc. RS Observation SWAP Crop Growth Model LAI, Evapotranspiration LAI, Evapotranspiration 4 . 00 4 . 00 Fitting 3 . 00 3 . 00 Assimilation by finding Optimized parameters By GA Eavpotranspiration LAI Evapotranspiration LAI 2 . 00 2 . 00 1 . 00 1 . 00 0 . 00 0 . 00 0 45 90 135 180 225 270 315 360 0 45 90 135 180 225 270 315 360 Day Of Year Day Of Year RS Model

6. Pixel pool Result pool Sequential Version • Take satellite image and loop through each pixel • For each pixel perform GA processing Sequential Version Best fit population

7. Challenges • The calculation time for identify SWAP parameters only for 1 pixel (1 sq.km) takes several minutes to 30 minutes. • Thus, a RS image of 1000 x 1000 sq.km of 1000x1000 pixels will take more than 50 years (30min x 1000 x 1000) is not acceptable. • Solutions • Parallel Computing on Cluster and Grid • Longitude: 100.008133 • Latitude: 14.388195

8. Thai Grid Infrastructure

9. Programming Model • Message Passing Using MPI • Widely adopted standard • Work efficiently on Cluster • Can work at some level on Grid • Grid RPC • Remote procedure call programming model on Grid system • GGF adopted as a standard programming on Grid • Several Implementation : GridSolve, Ninf-G

10. GridRPC Programming Model • Application partition into client/server • Multiple client can be used to achieve parallelism using asynchronous call • Role • Server side • Implement the algorithm as a remote procedure • Using Grid for security mechanism and execution management • Client side • Implement main workflow of the application • Handling input/output for user • Call Remote Functions at Server using GridRPC API

11. Ninf-G programming model • A software package which supports programming and execution of Grid applications using GridRPC • Advantages • Simple programming model that work efficiently over lossely copuple heterogenous system • Hiding most of the grid complexity from programmer and developer Grid application Grid RPC Low level Grid Middleware Grid Fabric Layer

12. Partitioning strategy on clusters and Grid • There are 3 possible partitioning strategies • Population Distribution • Pixel Distribution • Hybrid Distribution • Group compute nodes as virtual group • each virtual group take care one pixel • inside the group, distributing population to members • Pixel and population distributed has been implemented using MPI and reported in our previous work

13. Pixel pool Result pool Population Distribution Strategy GA Set of Population fitness Eval Eval Eval

14. Pixel pool Result pool Pixel Distribution Strategy Image Pixel Results GA GA GA

15. Pixel pool Result pool Hybrid Distribution Strategy Image Pixel Results Pixel Results GA GA population population Eval Eval Eval Eval Eval Eval Cluster Cluster

16. Extension to the Grid • Modify and adjust code to change distributing middleware from MPI to Ninf-G • Implement secure data-transferring method for Grid instead of using NFS for cluster environment • Gain benefit of Grid computing • More effective • More scalable • More security

17. Architecture

18. GA_SWAP(pixel_buffer, result) { Load(pixel_buffer); Run(swap); Send(result); }

19. Results & Discussions • Finish only pixel distribution over cluster • Experiment run on 3 clusters in the Grid • MAEKA 31 nodes Opteron Dual 1.4&1.8 GHz • AMATA 14 nodes Athlon XP 1 GHz • MAGI 4 Nodes Athlon XP 2500+

20. Result

21. Result

22. Result

23. Lesson Learned • Grid programming is still difficult • Learning curve • Debugging? • Grid reliability. May be useful to have a small development grid separate from deployment grid • Ninf-g application on grid is working fine. A good approach for development of grid applications.

24. Future Work • Improve performance • scheduler and node selecting(now is only round-robin) • Test running across international Grid • collaboration with Prof. Kento Aida,Titech • PRAGMA testbed • Deployment of ninf-g for Thai National Grid application development

25. Thank you Q&A