Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization

1. Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization D. Komagal Meenakshi (07MW05) PSG College Of Technology. Dr. G. Sudha Sadhasivam Asst. Professor Dept. of CSE. PSG College Of Technology.

2. Outline • Scheduling in Grid. • Problem Statement • Need For Job Grouping in Scheduling • PreviousWork Done in Job Grouping • Proposed System • Trust Based Filtering of jobs • Particle Swarm Optimization • Parallel PSO • Model for PPSO • Dynamic jobs • Results • Conclusion and Future work • Bibliography

3. Scheduling in Grid. Grid computing is a high performance computing environment to solve large scale computational demands. Task scheduling is a fundamental issue in achieving high performance in grid computing systems. Reason: Large numbers of tasks are computed on the geographically distributed resources, a reasonable scheduling algorithm must be adopted order to minimize job completion time with uniform load distribution.

4. Need An unorganized deployment of grid applications with a large amount of fine-grain jobs Leads to communication overhead dominate the overall processing time Low computation-communication ratio. Results

5. Need For Job Grouping in Scheduling Efficient job grouping-based scheduling system is required. A Grid Scheduler should Reduce the total transmission of user jobs to/from the resources. Reduce the overhead processing time of the jobs at the resources.

6. Jobs of an application [ fine grained ] Dynamically assemble • job groups [ coarse grained ] Transmit Grid resources Job Grouping

7. PreviousWork Done in Job Grouping • Comparison of Scheduling algorithms with and without job grouping. • In the context of DAG scheduling, grouping of jobs into clusters to reduce inter-job communication. • Job Grouping strategy, adaptive to run time environment • Job Grouping with PSO.

8. Proposed System • A novel job grouping method using Parallel PSO • To reduce the communication overhead. • Enhance the speed of completion of processes. • Improve resource utilization. • Improve parallel efficiency. • Uses PPSO to select the resources to minimize the make span. • Trust level and dynamism of jobs is considered • Tool Used - Gridsim-4.2-beta.

9. The Project aims at … • Job Grouping based on trust Using PPSO • Parallel Job Submission • Enhancing Computation-communication Ratio • Reducing The Overall Processing Time Of Jobs Using Parallelization • Improving Resource Utilization In The Grid Environment. • Trust based job filtering • Dynamic job submission

10. 1. Job Grouping Filtered Jobs of an application [ fine grain] based on Trust Dynamically assemble Using PPSO • job groups [ coarse grained ] Transmit to In Parallel Grid resources Grid resources Grid resources

11. Job Grouping User Input Total number of jobs Average MI rate of job Grid Resource File Granularity time Grid resource 0 MI deviation Percentage Overhead processing time Grid resource 1 Grid Resource Grid resource N Trust level Filter jobsbased ontrust (1) (2) (7) Gridlets Grid resources’ characteristics (3) Gridlet MI Resource MIPS Granularity time (4) Gridlet Scheduler Total MIPS (5) Grid resource 0 Grid resource 1 Grid resource 2 ….. Gridlet group 0 Gridlet group 1 Gridlet group 2 (6) Gridlet groups Resource IDs In parallel

12. 2. Trust Based Filtering of jobs • The Grid Information Service GIS gives the information about all the trust level of the resources . • The user submits the jobs with different trust values. • From this, the jobs that have trust values greater than the resource's trust value are filtered out. • Trust aware resource management and scheduling offer Quality of Service at application layer in grid environment.

13. 3. Particle Swarm Optimization • If large numbers of tasks are computed on the geographically distributed resources, a reasonable scheduling approach must be adopted in order to get the minimum completion time. • Task scheduling is a NP-Complete problem • Heuristic optimization algorithm can be used to solve NP-complete problems.

14. Particle Swarm Optimization (PSO) is an evolutionary optimization technique inspired by nature. • It simulates the process of a swarm of birds preying. • Its global searching ability can be used for neural network training, control system analysis and design, structural optimization. • It also has fewer algorithm parameters than genetic algorithm. • PSO algorithm works well on most global optimal problems.

15. PSO Concept • A swarm intelligence based algorithm finds a solution to an optimization problem in a search space. • Proposed solution exists in the form of a fitness function. • The swarm is typically modeled by particles in multidimensional space that have a position and a velocity. • A Particle is a candidate solution in the population and represents a task.

16. Particles fly through hyperspace . • An iterative process to improve candidate solutions is set in motion. The particles iteratively evaluate the fitness of the candidate solutions. • Particles posses two essential reasoning capabilities • Memory of their own best position and • knowledge of the global best of the swarm. • As the swarm iterates, the fitness of the global best solution improves. • All particles being influenced by the global best eventually approach the global best. This phenomenon is called 'convergence'.

17. PSO Algorithm • Initialize parameters • Initialize population randomly • Initialize each particle position vector and velocity vector Do { • Update each particle’s velocity and position; • Find a permutation according to the updated each particle’s position; • Evaluate each particle and update the personal best and the global best; • Apply the local search; • } While (!Stop criterion)

19. Parallel PSO Recent advances in computer and network technologies led to parallel optimization algorithms. Parallel PSO (parallel implementation of stochastic optimization alg)

20. Intialize # of particles f(x) f(x) f(x) # of iterations Check Convergence Update Parallel PSO design

21. Master RECEIVE INDIVIDUAL VALUE SEND GLOBAL VALUE Slave Slave Slave Model for PPSO

22. Trust based filtered Gridlet list Resource list Scheduler Call PPSO to assign Gridlet To Resources Create new grouped Gridlet With length= Total length Assign to resources Gridlet Grouping

23. 4. Dynamic jobs • Dynamic submission of jobs is considered. • User can submit jobs when other jobs are being processed. • The unused MIPS rating of the resources can be utilized in a efficient way such that grouping is done by considering the unused MIPS as total MIPS and the jobs are processed. • Then Parallel Submission of grouped Gridlets to resources is done

25. Simulation Time for Job Grouping using PSO vs. Parallel PSO

26. Total number of processed gridlets for different granularity time and resources

27. Load at resources during job grouping with PPSO

28. Difference in submission time of gridlets with PSO and PPSO

29. Add load balancing feature graph here

30. Conclusion • The proposed framework using PPSO has less simulation time compared to job scheduling framework using PSO as the simulation time is reduced. • Resource selection based on PPSO is used to generate an optimal schedule so as to complete the tasks in a minimum time than PSO as well as utilizing the resources in an efficient way. • Simulated results demonstrates load balanced resource selection. • Simulation results demonstrate that PPSO algorithm can get better effect for a large scale optimization problem.

31. Future Work • Future work would involve developing a more comprehensive job grouping-based scheduling system that takes into account QoS (Quality of Service) requirements of each user job before performing the grouping method. • Resource utilization can be done according to the capacity of the resource.

32. THANK YOU