self tuning energy aware ensemble model for server clusters
Download
Skip this Video
Download Presentation
Self-Tuning Energy-Aware Ensemble Model for Server Clusters

Loading in 2 Seconds...

play fullscreen
1 / 25

Self-Tuning Energy-Aware Ensemble Model for Server Clusters - PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on

Self-Tuning Energy-Aware Ensemble Model for Server Clusters. Team Members : Sanket Dangi (MT2009032) Team Lead Gaurav Kapoor (MT2009059) Deepthi Karnam(MT2009060) Sudha Mani (MT2009073) Celina Madhavan (MT2009075). Architecture. Master. Clients. Server Cluster.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Self-Tuning Energy-Aware Ensemble Model for Server Clusters' - hasad


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
self tuning energy aware ensemble model for server clusters

Self-Tuning Energy-Aware Ensemble Model for Server Clusters

Team Members :

SanketDangi (MT2009032) Team Lead

GauravKapoor(MT2009059)

Deepthi Karnam(MT2009060)

SudhaMani(MT2009073)

Celina Madhavan(MT2009075)

architecture
Architecture

Master

Clients

Server Cluster

Reference :Energy Efficient Real Time Heterogeneous Server Clusters by CosminRusu,AlexendreFerrerira,ClaudioScordino,AaronWatson,Rami Melham

steps to achieve goal according to development plan
Steps to achieve goal according to Development plan
  • Identifying User processes…………………………..………..……DONE
  • Identifying CPU idle % and send system to hibernation if it crosses some threshold………………………………………………….…….DONE
  • Setting up wakeup alarm in BIOS……………………………….…DONE
  • WakeonLan Configuration…………………………………………..DONE
  • Energy Consumption Calculation in different modes…………DONE
  • Port Filtering and Storing Traffic in Database……………….…DONE
  • Traffic Pattern Recognition Algorithm-kNN Algorithm……….DONE
  • Hibernation Decision depending on Traffic Pattern and Energy Consumption Calculation………………………………YET TO BE DONE
  • Scalability of the Solution………………………………YET TO BE DONE
experimental setup to test the various modules developed
Experimental Setup to Test the Various Modules Developed

THIS LAPTOP ACTS AS SERVER 2

AND IS SENT INTO HIBERNATION

AND WAKES UP WHEN A CERTAIN

NUMBER OF PACKETS ARE FORWARDED

TO IT FROM THE MASTER

THIS LAPTOP ACTS AS SERVER 1

WHICH COLLECTS THE PACKETS GENERATED

BY THE CLIENT AND FORWARDED BY

MASTERAND DISPLAYS THE COUNT OF PACKETS

GENERATED

ROUTER

THIS LAPTOP ACTS AS A CLIENT

FROM WHICH THE TRAFFIC IS

GENERATED

THIS LAPTOP ACTS AS THE MASTER

AND GETS THE REQUESTS FROM THE CLIENT

AND FORWARDS IT TO THE SERVERS

deliverable 1a b identifying number of user processes and idle cpu utilization
Deliverable 1a,b :Identifying Number of User Processes and Idle CPU Utilization
  • If there is no client processes running in the server, then the server can be sent to hibernation.
  • Identifying CPU Idle % and send system to hibernation using ACPI hibernation script if it crosses some threshold limit
  • Shell Script is used
  • Commands Used:
  • ps
  • grep
  • wc
  • mpstat

Process

(Number of User Process in server)

Process

(Identifying Idle CPU %)

Master

deliverable 1c wakeup alarm
Deliverable 1c :Wakeup Alarm
  • 1.Depending upon Historical Pattern, no. of active servers would be decided and wakeup alarm will wakeup server(s).
  • 2.Alarm is set in BIOS. The alarm IRQ (interrupt) is set
  • 3.Shell Script is used
  • 4.Files used:
  • “/sys/class/rtc/rtc0/wakealarm”
  • “/proc/driver/rtc”

At set alarm time the servers wakeup

Master

Servers

deliverable 2 testing deliverables on client server architecture
Deliverable 2 :Testing Deliverables on Client Server Architecture

Server

Master

Key generated

  • To establish Communication between Master and Server :
  • 1. SSH configuration
  • 2. PasswordLess Login by Generating RSA Key using ssh-keygen command
  • Testing of Previous Deliverables on client server architecture
deliverable 3 wakeup on lan
Deliverable 3:Wakeup On LAN

Active

Server

No more requests can be serviced by the active server

Master

Client1

Requests

Client n

When active server can’t service anymore requests another server is activated using WOL

Server

deliverable 3 wakeup on lan continued
Deliverable 3:Wakeup On LAN(Continued)
  • Assumptions: Wired LAN
  • Package Used: wakeonlan
  • Command Used:

wakeonlan

deliverable 4a energy measurements
Deliverable 4a :Energy Measurements
  • Energy Meter Specification:
    • MACO EM 08 single phase multifunction meter 1mpA 230V
  • The multifunction meter was used to measure the power consumed by a laptop in watts during normal functioning, sleep mode, hibernation mode, and shutdown.
  • The energy consumed in watthrs by the laptop while going into hibernation and waking up was also measured.
  • Based on the measurements we have arrived at the following conclusions
      • The laptop consumes less than a watt in sleep and hibernation modes. This was measured as 3W and 1W for a desktop PC.
      • To save energy it is advisable to keep a server in idle mode for a period of 1.4 mins when a laptop is configured as a server rather than send it to hibernation.

V

A

KW

KWHr

MULTIFUNCTION

METER

Power

socket

deliverable 4b port filtering and storing traffic in database
Deliverable 4b:Port Filtering And Storing Traffic In Database

If any request hits, increment RequestCounter

Client 1

80

Wait for Fixed Duration(15 mins)

Store RequestCounter in Database

20

Client 2

Ports

Reset RequestCounter

Clients

Master

deliverable 4b port filtering and storing in database continued
Deliverable 4b:Port Filtering And Storing In Database(Continued)

Packages Used: JDK6,MySql5.1,packETH(traffic generator)

Libraries Used: jpcap library

Procedure:

1.Open Ethernet device for sniffing

2.Generate Random Traffic through packETH

3.Capture Packets and Count for every fixed duration

4.Store count in database corresponding to that duration

deliverable 4c pattern recognition algorithm knn algorithm
Deliverable 4c:Pattern Recognition Algorithm-kNN Algorithm
  • The data mining algorithm uses a pattern recognition technique proposed by Taehyung Kim, Hyoungsoo Kim, Cheol Oh and Bongsoo Son in their paper Traffic Flow Forecasting Based on Pattern Recognition to Overcome Memoryless Property available at http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=04197439
  • Since traffic flow is not completely random in nature, there should be some patterns in which the past traffic flow repeats itself. This pattern recognition technique, enables us to consider the past sequences of traffic flow patterns to predict the future state.
  • Consider a discrete time-series q= {q1, q2, ..., qn} (e.g., 5 min. traffic volumes) where n is the total number of points in the series.This is our available historical data for say N days, which forms an estimation set.
  • The algorithm works as follows. We define a pattern size ‘l’ and search for the pattern defined by {qn-l,qn-l+1, ….qn} in the historical data for a maximum number of ‘k’ nearest neighbours.
  • If a match is found at qj we predict the value of qn+1 by taking the average of qj+1 for all the ‘k’ nearest neighbours.
deliverable 4c pattern recognition algorithm knn algorithm1
Deliverable 4c:Pattern Recognition Algorithm-kNN Algorithm
  • The algorithm first takes a minimum value of ‘k’ and ‘l’ and predicts the traffic for each qn+1 in the estimation set based on the available qn values
  • The root mean squared error (RMSE) between the actual and predicted values, for these choices of neighborhood size and pattern size,is calculated for the entire estimation set.
  • The process is repeated incrementing ‘l’ upto a maximum size of lmax
  • The process is again repeated incrementing values of ‘k’ upto a maximum of kmax.(kmax and lmax are chosen by us)
  • The optimum values of ‘k’ and ‘l’ for the particular network is arrived at by plotting ‘l’ versus RMSE for various ‘k’.
  • These optimum values of ‘k’ and ‘l’ are then used for predicting the pattern for the (N+1)th day.
  • The pattern for the (N+1)th day is stored in a database and used by the load balancing algorithm to take decisions on sending the servers to hibernation.
slide19
TRAFFIC STATISTICS FROM GERMAN INTERNET EXCHANGEhttp://www.de-cix.net/content/network/Traffic-Statistics.html

MATCHING PATTERNS

flow chart
FLOW CHART

START

Increment ‘k’

SET k=1,l=1

For each qn match the

pattern{qn,….qn-l} with

‘k’nearest matches in

historical data (N days)

is ‘k’

yes

no

Find qn+1 based

on the available

qn values for each

qn in the estimation set

Plot graph of ‘l’

versus RMSE

and find optimum

value of ‘k’ and ‘l’

Calculate RMSE

Predict graph for

(N+1)th day

Increment ‘l’

yes

END

is l< lmax?

no

deliverable 4c request forwarding
Deliverable 4c:Request Forwarding

Web

Server 1

Client1

HTTP Request coming to Master Server is forwarded to Web Server 1

Master

Client 2

Requests

Web Server 2

If requestcounter exceeds capacity of webserver 1, WOL signal sent and further requests are forwarded to it

Client n

deliverable 4c request forwarding continued
Deliverable 4c:Request Forwarding (Continued)
  • Iptables used
  • Start port forwarding by editing /etc/sysctl.conf file
  • net.ipv4.ip_forward=1
  • For port forwarding,use command
  • iptables –t nat –A PREROUTING –p tcp –d --dport -j DNAT –-to :
work to be done
Work To Be Done

Decision Regarding whether to send a system into hibernation or not?

While sending server to hibernation, historical traffic pattern and energy consumption reading would be check and decision regarding sending it to hibernation or not would be done.

b) Scalability of solution

Try the solution for variable and Heterogeneous servers.

ad