1 / 19

Cluster Computing for SWAP Crop Model Parameter Identification using RS

Cluster Computing for SWAP Crop Model Parameter Identification using RS. HONDA Kiyoshi Md. Shamim Akhter Yann Chemin Putchong Uthayopas. Importance of Crop model parameters identification. Agriculture Activity Monitoring Sowing date, Cropping intensity, Growth, Water stress, Yield, etc.

essien
Download Presentation

Cluster Computing for SWAP Crop Model Parameter Identification using RS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cluster Computing for SWAP Crop Model Parameter Identification using RS HONDA Kiyoshi Md. Shamim Akhter Yann Chemin Putchong Uthayopas

  2. Importance of Crop model parameters identification • Agriculture Activity Monitoring • Sowing date, Cropping intensity, Growth, Water stress, Yield, etc. • Production for Food Security • Water Management in Irrigation Activity • Remote Sensing is quite useful data, however modeling has not been done. – empirical model • Crop growth model ( SWAP ) • Continuous monitoring in various aspects • Prediction • Spatial Parameter estimation & Calibration ->RS • Data Assimilation Technique • To estimate parameters which cannot be observed by RS • High-Resolution Remote Sensing • Temporal Info -> Low Resolution Remote Sensing; Mixed Pixel

  3. SWAP Model Diagram Adopted from Van Dam et al. (1997) Drawn by Teerayut Horanont (AIT)

  4. SWAP Model Parameter identification - Data Assimilation using RS and GA - SWAP Input Parameters sowing date, soil property, Water management, and etc. RS Observation SWAP Crop Growth Model LAI, Evapotranspiration LAI, Evapotranspiration 4 . 00 4 . 00 Fitting 3 . 00 3 . 00 Assimilation by finding Optimized parameters By GA Eavpotranspiration LAI Evapotranspiration LAI 2 . 00 2 . 00 1 . 00 1 . 00 0 . 00 0 . 00 0 45 90 135 180 225 270 315 360 0 45 90 135 180 225 270 315 360 Day Of Year Day Of Year RS Model

  5. Field photos • Longitude: 100.008133 • Latitude: 14.388195

  6. Estimated parameters • DOYCrop1 = 19 • DOYCrop2 = 188 • Crop.Int.Crop2 = 0.32 • Fitness = 4.537 • Generation found = 31 (popsize=5) • Calculation time approximate 15 minutes

  7. A Practical Problem Arises • A practical issue arises with the overall calculations time load for assimilating data with remote sensing data. • The calculation time for identify SWAP parameters only for 1 pixel (1 sq.km) takes several minutes to 30 minutes. • Thus, a RS image of 1000 x 1000 sq.km of 1000x1000 pixels will take more than 50 years (30min x 1000 x 1000) is not acceptable. • Solutions • LUT Method • Segmentation • Parallel Computing

  8. Objectives • Implement SWAP-GA model in cluster computers • Propose and evaluate job distribution models • Population by Population • Pixel by Pixel • Hybrid • Develop a model to estimate calculation time

  9. Clusters Technical Specifications • 3 Clusters (AIT and Kasetsart universities Clusters) • Optima 9 nodes: Front-end: Athlon XP 1800+ 512 MB RAM 80 GB IDE Disk Compute Node: Athlon Xp 1800+ 512 MB RAM 40 GB IDE Disk Disk Interconnection: Fast Ethernet http://optima.ait.ac.th/scmsweb/scms_home.html • Magi 4 nodes: Athlon XP 2500+ 512 MB RAM 80 GB Hard Disk Disk Interconnection: Gigabit Ethernet Interface http://magi.cpe.ku.ac.th/scmsweb/scms_home.html

  10. Technical Spec Continue.. Maeka 32 nodes: Front-end: Dual AMD Opteron 1400 MHz RAM PC2100 : 6GBs Disk SCSI Ultra 320: 72GBs 15 machines Dual AMD Opteron 1400 MHz RAM PC2100 : 3GBs Hard Disk SCSI Ultra 320: 72GBs 16 machines Dual AMD Opteron 1800 MHz RAM PC2700 : 3GBs Hard Disk SCSI Ultra 320: 72GBs Gigabit Ethernet Interface2 http://maeka.ku.ac.th/spec.html

  11. Implementation Schemes • 3 Implementation Schemes are proposed • Population by population (Distributing population) • Pixel by pixel • Hybrid (combination of Population by population and pixel by pixel)

  12. MASTER SWAP-GA 1PIXEL SLAVE n n=popsize Population n ./SWAP SLAVE 1 Population 1 ./SWAP . . . . . . . Distributing Population by Population • Each Generation have n populations to be evaluated • 1 population ( 1 set of parameters ) will be distributed to 1 slave. If n is more than slave, more than 1 population will be distributed to 1 slave. • Slaves do the evaluation, generate fitness and send back the population (with fitness) to Master. Master Next Generation Distribute Population Without Fitness Slave • Read Input Files • Run SWAP • Analyze Result • Population With Fitness

  13. MASTER SLAVE 1 1 PIXEL SWAP-GA SLAVE n n=10 1 PIXEL SWAP-GA Nodes Distributing Pixel by Pixel • All pixels will be distributed among the available slaves. • Each slave will evaluate total serial SWAP-GA procedure inside itself for one pixel at a time and produce a total assimilation result for that pixel in a file in their local Hard Disk Master Assimilation result Distribute Pixel Slave • All Populations Without Fitness • Read Input Files • Run SWAP • Analyze Result • All Populations With Fitness Next Generation

  14. 0 4 3 0 3 7 1 5 6 2 1 4 5 2 Slave 1 1 PIXEL SWAP-GA Slave n 1 PIXEL SWAP-GA . . . Local Master 1 PIXEL SWAP-GA Slave n n=popsize Population n ./SWAP Slave 1 n=popsize Population n ./SWAP . . . Hybrid (Population by Population and Pixel by Pixel Together) Master Here Node 0,3 are Master Node 1,2,4,5 are Slaves Pixel1 Pixel2 pop1 pop2 pop1 pop2 6 Available Nodes Here Node 0,4 are Master Node 1,2,3,5,6,7 are Slaves 8 Available Nodes

  15. Hardware Architecture Influence on Speed up :( Pop & Pop: 1 SWAP on 1 Slave ) Copy File Run 1 SWAP • Distribution Population Method • Copy all necessary (inputted) files (CF)=336KB, Read files (RF)=.0067KB, Run SWAP executable file (RS)=32KB , Write files (WF)=1.46KB • The parameters inside parallel SWAP-GA are fixed in all Clusters (No of population=2, No of Generation=1, PXOVER=0.8, PMUTATION=0.25) The Bandwidth Times for Copying Files and Executing SWAP According to above Table it is clear that Hardware Architecture Influence on Speed Up.

  16. Software Architecture Influence on Speed up:( Total Throughput / n-SWAP : sec/SWAP) Average SWAP Execution Time (in second) on Different Clusters • Here, a comparison between the running time (in second) of a single SWAP execution (Total Model running time / Total No. of SWAP execution) on these Clusters (by guessing that the I/O factors and GA running factors are taking constant time in all cases). • Parameters in Optima, Maeka and Magi: • -No of pixel (9), No of Population (10), No of generation (20). • -9 processors are used in Optima and Maeka but due to reason that • Magi is a 4 nodded Cluster so, 4 processors are used in Magi. Hybrid model is given best performance (time domain) among the three predefined models

  17. Compare Serial SWAP_GA with Clustering SWAP-GA Pop by Pop model n Slaves = n Pop. Serial Cluster Figure: Timing Diagram Serial SWAP-GA and Parallel SWAP-GA by Distributing Population in Optima Pix by Pix model n pixel = n slave Figure: Timing Diagram Serial SWAP-GA and Parallel SWAP-GA by Distributed Pixel in Optima

  18. Scalability for Pop by Pop model ( Optima: n Slave = n Pop, 1 pix ) For i belongs to population, j belongs to generation T =Agj Pi + Cgj (sec) (Serial SWAP-GA) Agj= 11.16 gj – 6.94 Cgj =-1.385.gj + 3.433 (Parallel SWAP-GA) Agj= 0.865gj + 1.22 Cgj =5.57 gj – 1.93 The symbols, Pi is the population number, T is the time in population pi and generation gj. Agj and Cgj are find out with respect to the value of gj and used as constants to find out the value of T. • However, within this experiment highest 25 generations and 8 populations are used but all the above equations have given a generic model. • For any value of generation and population number the running time in both parallel SWAP-GA and serial SWAP-GA can be measured by using these equations. • The above equations can solve the complication of running parallel SWAP-GA within Clusters those hold limited number of slaves (PC).

  19. Conclusion & Future Work • 3 Implementation Models were proposed and evaluated on 3 clusters • Hybrid model always gives the fastest result • A Model to estimate calculation time on Optima was established. Models in different model will be developed. • Improve Time Measurement • Implementation for bigger clusters ( new cluster in Kasetsrat, Titech ) • A monitoring System ( CPU load ) • Finding idling CPU to send jobs : scheduler • Implementation to Thai GRID. • A fully SWAP-GA software module • Package, documentation, scripting. • We believe that this research will include one new part not only in the Grid-Cluster computing arena but also in the Remote Sensing image analyzing field.

More Related