capacity planning and workload forecasting
Download
Skip this Video
Download Presentation
Capacity Planning and Workload Forecasting

Loading in 2 Seconds...

play fullscreen
1 / 105

Capacity Planning and Workload Forecasting - PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on

Capacity Planning and Workload Forecasting. Definitions (1). Capacity – the (maximum) amount that a device is able to tolerate Work Throughput Storage Capacity Plan – a strategy to effectively utilize the capacity of a device

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Capacity Planning and Workload Forecasting' - giorgio-lunney


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
definitions 1
Definitions (1)
  • Capacity – the (maximum) amount that a device is able to tolerate
    • Work
    • Throughput
    • Storage
  • Capacity Plan – a strategy to effectively utilize the capacity of a device
  • Capacity Planning – the determination of the predicted future time when system saturation will occur
  • Saturation – that point in time when a computer system is asked to do more work than it is capable of
  • Device Saturation – that point in time when a particular device within a computer system is asked to do more work than it is capable of
  • Work – the transfer of one (1) byte (of data) between a processor and a storage device
  • Workload – the amount of work that is being performed by a device or system
  • Workload Management – the strategy for effectively utilizing the performance of work in a computer system

Capacity Planning and Workload Management

definitions 2
Definitions (2)
  • Background – processing that occurs “behind the scenes” and (normally) does not involve direct (human) user interaction
  • Foreground – processing that involves direct interaction between (human) users and computers
  • Planning Horizon – a period of time for which a strategy and associated objectives are established
    • Long range
    • Medium range
    • Short range
  • Initial Load – the amount of resources that are required at the time an information solution is deployed
  • Growth Factor – the rate of change (Δr/Δt) in the amount of resource used over a planning horizon

Capacity Planning and Workload Management

definitions 3
Definitions (3)
  • Performance Constraints – business rules that are imposed upon the responsiveness of an information solution
  • Simulation – a(n) (parameter- driven,) algorithmic approach to modeling the (expected) performance of an information solution
  • Priority Level – the level that an executing program is serviced by the computing environment

Capacity Planning and Workload Management

capacity planning schematic
Capacity Planning Schematic

Desired

Service

Levels

Workload

Evolution

System

Parameters

Capacity Planning

Cost-effective

Alternatives

Saturation

Points

Capacity Planning and Workload Management

Source :Menasce, Almeida, & Dowdy, Capacity and Performance Modeling, ISBN 0-13-789546-1

information technology components to consider
Networks – communications paths

Processor – instructions executed

Memory – primary storage

Files – secondary storage

Work

Sort

Storage

Libraries

Databases – structured permanent storage

Topology – nodes in the network

Security – information protection Software

Executive

Applications

Foreground

Background

Information Technology Components to Consider

Capacity Planning and Workload Management

critical questions
Critical Questions
  • What is the overall architecture of the current environment?
    • What is in place?
  • What is the this environment capable of?
    • What can it do?
  • What architecture is needed?
  • What will it cost?

Capacity Planning and Workload Management

importance of capacity planning
Importance of Capacity Planning
  • User satisfaction
  • Image of the enterprise
  • Productivity considerations
  • Budgetary considerations
  • Risk
  • Control

Capacity Planning and Workload Management

capacity planning and workload management relationships
Capacity Planning and Workload Management - Relationships

Strategic Plan

Technology

Plan

Capacity Plan

Business Vision and Mission

Tactical Plan

Operational

Plan

Workload

Management

Capacity Planning and Workload Management

centralized versus decentralized is facilities

λ

λ

λ

λ

user1

user1

user1

user1

λ

λ

λ

λ

user2

user2

user2

user2

Centralized versus Decentralized IS Facilities

Batch

Central

Facility

Centralized

Decentralized

Capacity Planning and Workload Management

Source :Menasce, Almeida, & Dowdy, Capacity and Performance Modeling, ISBN 0-13-789546-1

capacity planning methodology
Capacity Planning Methodology
  • What is the current installed capacity?
  • What future services should be provided?
  • What are the planned quality goals?
  • What is the most cost effective configuration to meet the state goals?

Capacity Planning and Workload Management

types of environments
Types of Environments
  • OLTP – Online Transaction Processing
    • Transactional activities
    • Traditional “business” transactions
  • OLAP – Online Analytical Processing
    • Algorithmic activities
    • Search and/or computation intensive
  • Centralized processing
    • Single node
  • Distributed processing
    • Multi-node

Capacity Planning and Workload Management

capacity planning process
Capacity Planning Process

IT

Environment

A

Alternative A

Alternative B

Alternative C

Capacity

Planning

Process

Business Plans

B

C

Product

Lines

Capacity Planning and Workload Management

model based capacity planning
Model-based Capacity Planning

Workload

Model

Corporate

Plans

Understand

Environment

Characterize

Workload

Validate and

Calibrate

Performance

Model

System

Alternatives

Predict

Performance

Forecast

Workload

Validated

Model

Capacity Planning and Workload Management

building the capacity plan 1 understand the current environment
Hardware

One or more computers

Share common resources

Networks

Interconnect different computers

Operating System

Interacts directly with hardware

Provides services to other software

Isolates software from hardware anomalies

Support Software

Application Programs

Operational

Developmental

Application

Application

Development

DBMS Transaction Command

Support Software

Building the Capacity Plan1. Understand the Current Environment

Operating System

Service Request

Hardware

Capacity Planning and Workload Management

building the capacity plan 2 understanding capacity a
Building the Capacity Plan2. Understanding “capacity” (a)
  • Theoretical Capacity (of a computing system) – maximum rate at which such a system can perform work
  • Throughput (of a computing system) – the rate at which such a system services requests made of it
  • Effective Capacity (of a computing system) – the largest throughput at which response time remains acceptable
    • Responsiveness limits the amount of effective work processed by a computing system

Capacity Planning and Workload Management

building the capacity plan 2 understanding capacity b

Theoretical Capacity

1.0

0.8

0.6

0.4

0.2

Response Time ( R )

Acceptable Level of Response

Effective Capacity

2 4 6 8 10 12

Throughput (tps)

Building the Capacity Plan2. Understanding “capacity” (b)

Capacity Planning and Workload Management

building the capacity plan 3 computing system components
Building the Capacity Plan3. Computing System Components

NETWORK

Storage

Memory

I/O

Processor(s)

Capacity Planning and Workload Management

building the capacity plan 4 computing system metrics
Building the Capacity Plan4. Computing System Metrics
  • Cycle time – the speed of a processor
    • Length  10-9 seconds
    • Rate  MHz or GHz
    • MIPS – millions of instructions/second = instructions/cycle * cycles /sec * 10-6
  • Benchmarks – standard programs used to measure component performance
    • TPA – network performance
    • TPB – database system performance
    • TPC – OLTP programs

Capacity Planning and Workload Management

building the capacity plan 5 estimating magnetic disk performance
Building the Capacity Plan5. Estimating Magnetic Disk Performance
  • Consists of several elements
    • Seek time – time required to move the disk arm to the destination track
    • Rotational delay – time required to align the target coordinate with the read/write head
    • Transfer time – the time required to move the requested data bits
    • Controller time – the amount of time (overhead) used by the controller
  • Average Access Time = average seek + average rotational delay+ (block size/transfer rate) + controller time

Capacity Planning and Workload Management

building the capacity plan 6 time windows a

Peak

Valley

Activity Volume

Processing Time Period

Building the Capacity Plan6. Time Windows (a)
  • Consideration of when work occurs
    • Peaks
    • Valleys

Capacity Planning and Workload Management

building the capacity plan 6 time windows b
Building the Capacity Plan6. Time Windows (b)

What is this graph telling ?

Activity Volume

S M T W R F S

S M T W R F S

S M T W R F S

S M T W R F S

Week 1 Week 2 Week 3 Week 4

Capacity Planning and Workload Management

building the capacity plan 6 time windows c
Building the Capacity Plan6. Time Windows (c)
  • Granularity
    • Determining the smallest meaningful unit of time

Capacity Planning and Workload Management

building the capacity plan 6 distribution considerations
Building the Capacity Plan6. Distribution Considerations
  • Every node in the network must be evaluated in a like manner

Activity Volume

Time Units

Node

Capacity Planning and Workload Management

building the capacity plan 7 service levels
Building the Capacity Plan7. Service Levels
  • Quality of the computing services
  • Contractual
    • Cost versus benefit
    • Competition/business conditions
    • Mission criticality
    • Previous experience
  • Perception
  • Generally accepted measures
    • MTBF – mean time between failures
    • MTTR – mean time to repair
    • Availability =(interval length – downtime)*interval length-1

Capacity Planning and Workload Management

building the capacity plan 8 components

Application

Application

Development

DBMS Transaction Command

Support Software

Building the Capacity Plan8. Components
  • Everything that executes on a computer does work and contributes to total workload
    • When executing
    • Where executing
  • So, at any moment in time
    • The workload varies
  • Capacity plan must project

Operating System

Service Request

Hardware

Capacity Planning and Workload Management

building the capacity plan 9 component and calculation
Building the Capacity Plan9. Component and calculation
  • Calculate on an application by application basis
  • Consider each of the supporting services used
  • Consider nodal implications
  • Aggregate and plot
  • Adjust to account for unanticipated situations
    • At any point, does the estimated aggregate exceed the root-mean-square of total capacity (0.707)?
  • What risks have been identified

Capacity Planning and Workload Management

the workload model
The Workload Model
  • Representation of actual workload being studied
  • Two categories
    • Synthetic models
      • Combination of actual and specially developed components
    • Artificial models
      • Special purpose programs
      • Descriptive parameters

Capacity Planning and Workload Management

synthetic models
Synthetic Models
  • Experientially based
  • Elements/components
    • Kernels – key programs from existing workload
    • Synthetic Programs – special purpose programs intended to develop critical measurands

Capacity Planning and Workload Management

artificial model
Artificial Model
  • Executable
    • Suite of computer programs
    • Instruction mixes –
      • Hardware demonstration programs
      • Test computational and I/O components
    • Kernels –
      • Code elements from computationally-intensive programs
      • Measure processor without I/O components
    • Synthetic Programs
      • Devised code – places demand on different computing system resources
      • Do not resemble real workload
      • Different from benchmarks
  • Non-executable
    • Parametric representation of workload resource components

Capacity Planning and Workload Management

artificial model parameters
Artificial Model Parameters
  • Typical parameters
    • Program interarrival time
    • Service demand
    • Program size
    • Execution mix
      • Classes of programs
      • Corresponding levels of multiprogramming
  • Processor factors
    • System load
    • Device capability
    • File reference location
  • Derived parameters
    • Frequency distribution of requests
    • Request interarrival time distribution
    • File referencing behavior
    • Size of read and write requests

Capacity Planning and Workload Management

using the artificial model
Using the Artificial Model
  • Capture key aspects of a computer system
  • Relate key aspects
    • Mathematical formulae
    • Computational algorithms
  • Require
    • Workload intensity
    • Service demand
  • Limited representation requires simplifying assumptions

Capacity Planning and Workload Management

workload characterization methodology
Workload Characterization Methodology
  • Identification of basic components
  • Choice of Operating Parameters
  • Data collection
  • Partitioning the workload
    • Resource usage
    • Applications
    • Geographical orientation
    • Functional classes
    • Organizational units
    • Class type
  • Calculating class parameters
    • Averaging
    • Clustering
    • Data analysis
    • Scaling techniques
    • Clustering algorithms

Capacity Planning and Workload Management

identifying basic component of work
Identifying Basic Component of Work
  • Job
  • Transaction
  • Command
  • Request
  • Result of this step 
    • What does the workload under study consist of?

Capacity Planning and Workload Management

choice of characterizing parameters
Choice of Characterizing Parameters
  • Component representation
    • Workload intensity
      • Arrival rate
      • Number of terminals
      • Think time
      • Number of simultaneously executing programs
    • Service demands
      • Specified by k-tuple (Di1, Di2, … , Dij)
        • K  number of resources considered
        • Dij  service demand
        • i basic component identifier
        • j  resource identifier

Capacity Planning and Workload Management

data collection
Data Collection
  • Assigns values to parameters of each model component
  • Generates same number of characterizing tuples as number of workload components
  • Included tasks
    • Identify time windows that define measurement sessions
    • Monitor and measure system activities during defined sessions
    • Assign values to all characterizing parameters

Capacity Planning and Workload Management

partitioning the workload
Partitioning the Workload
  • Actual workloads – collection of heterogeneous component
  • Single class representation lack accuracy
  • Goal – translate heterogeneity into homogeneity
    • Divide workload into classes whose populations are homogeneous components

Capacity Planning and Workload Management

partitioning the workload1
Partitioning the Workload
  • Resource usage – classify components based on common use of critical resources
  • Applications – classify components based on class of applications
    • Subjectization of the enterprise
  • Geographical orientation – classify components based on where they execute
  • Functional classes – classify components based on the computational functions being serviced
  • Organizational units – classify components based on the organizational structure of an enterprise
    • Consider organizational culture
  • Type – classify components based on processing mode and associated descriptive parameters
    • Interactive (terminal)
    • Transaction
    • Batch (background)

Capacity Planning and Workload Management

calculating class parameters averaging
Calculating Class Parameters - Averaging
  • Homogeneous components
  • Represent a specific value by the average for the class
  • Consider the tuple (Di1, Di2, … , Dik) whose arithmetic mean for each parameter is expressed asDj = p-1Σ Dij j = 1,2, … , k

ki= 1

Capacity Planning and Workload Management

calculating class parameters clustering
Each clear bullet represents a transaction execution

Resource oriented

Black bullets represent resultant for a class (Ci)

Statistical description

Calculating Class Parameters - Clustering

C1

CPU TIME

C2

C3

I/O TIME

Capacity Planning and Workload Management

calculating class parameters data analysis
Set of activities

Sampling

Parameter transformation

Outlier removal

Distribution options

Histogram

Logarithmic

Sampling

Specific

Random

Parameter transform

Representation in a standard model

Outlier removal

Elimination of extremes

Calculating Class Parameters – Data Analysis

Capacity Planning and Workload Management

calculating class parameters data analysis and distance

μ

-2σ

-1σ

X

CPU TIME

I/O TIME

Calculating Class Parameters – Data Analysis and Distance

Distance

C1

CPU TIME

C2

C3

I/O TIME

Data Analysis

Capacity Planning and Workload Management

calculating class parameters scaling
Calculating Class Parameters – Scaling
  • Used to avoid problems arising from parameters with very different values and ranges
  • Use a z-distribution for each parameter z = measured value – mean value standard deviationwhere the mean and standard deviation are standard statistical measures

Capacity Planning and Workload Management

mean and standard deviation formulae
Mean and Standard Deviation Formulae
  • Meanx = Σ xi
  • Standard Deviationσ = (Σ(xi – x)2)/(n – 1))1/2
  • Variance  σ2

ni = 1

n

i = 1

Capacity Planning and Workload Management

calculating class parameters clustering algorithms
Minimal spanning tree

Hierarchical algorithm that starts by considering each component of a workload to be a cluster

Algorithm

Set initial number of clusters = number of components

Determine parameter values of each cluster

Calculate the inter-cluster distance matrix

Determine the minimum non-zero element

Decrement number of clusters

Repeat steps 2 – 5 as necessary

K-means algorithm

Non-hierarchical clustering technique based on finding k points in the workload.

Algorithm

Set the number of clusters to k

Chose k starting points as initial estimates

Examine each point in the workload; allocate it to nearest cluster; recalculate cluster’s position

Repeat step 3 as needed until no point changes its cluster assignment or a maximum number of passes is performed

Calculating Class Parameters – Clustering Algorithms

Identifying natural groups of components based on resource requirements

Capacity Planning and Workload Management

workload forecasting
Workload Forecasting
  • Computer workloads change as the “business” changes
    • New applications
    • Changes to existing applications
    • Increase in processing volumes by existing applications
    • Environmental enhancements
  • Difficulties
    • Obtaining reliable information
      • Terminology and communications with users/customers
      • Needs of the “under construction” applications
    • Latent demand – deferred processing requests

Capacity Planning and Workload Management

the role of business
The Role of Business
  • Technical requirements = f (business requirements)
  • Computing capacity = f (business base, strategic planning)
  • Critical applications account for majority of computer resource utilization
    • What are they?
    • Use to identify
      • KVI – Key Value Indicator
      • NBU – Natural Business Unit
      • FBU – Forecasting Business Unit

Capacity Planning and Workload Management

workload forecasting methodology
Workload Forecasting Methodology
  • Select applications to be forecasted
  • Identify business units associated with applications whose growth will be forecasted
  • Summarize the statistics
  • Collect statistics concerning business units being chosen
  • Translate business units into computer demands
  • Forecast future resource demand as a function of the business units.

Capacity Planning and Workload Management

forecasting techniques
Forecasting Techniques
  • Moving average
  • Exponential smoothing
  • Regression methods

Capacity Planning and Workload Management

moving average
Moving Average
  • Simple forecasting technique
  • Allows value for next period to equal average of a set of previous observations
  • High reliability for (near) steady-state situations
  • Only one forecast value can be generated at a time
  • Forecast value calculationft+1 = (yt + yt-1 + . . . + yt-n+1 )/n
  • Error calculation – mean squared error (MSE) MSE = Σ (yt – ft )/n

n t = 1

Capacity Planning and Workload Management

exponential smoothing
Exponential Smoothing
  • Similar to moving average
  • Greater emphasis on most recent data
  • Exponential smoothing formulaft+1 = ft + α (yt – ft )where ft+1 is the forecast for period t+1yt is the actual value at time tα is the smoothing weight 0 < α < 1

Capacity Planning and Workload Management

regression methods
Regression Methods
  • Used to estimate the value of one variable as the function of several others
  • Simple linear regression – y = a + bxwhere b = ( Σ xiyi – nxy)/( Σ xi2 – nx2 ) a = y – bx y = n-1Σ yi x = n-1Σ xi n = number of observations

n i = 1

n i = 1

n i = 1

n i = 1

Capacity Planning and Workload Management

performance prediction techniques

Rules of

Thumb

Trend

Analysis

Performance Models

Benchmarks

Analytical

Simulatiion

Performance Prediction Techniques
  • Rules of Thumb
    • I/O channel utilization < 35 %
    • CPU utilization < 80 %
    • Peak utilization < 90%
    • Average utilization < 40%

Low Complexity and Cost High

Capacity Planning and Workload Management

workload and capacity planning in conclusion
Workload and Capacity Planning in Conclusion
  • Understand the (current) environment
  • Specify the service levels
  • Perform workload characterization; generate a workload model
  • Develop a performance model
  • Validate and calibrate the system model
    • Workload model
    • Performance model
  • Forecast workload growth
  • Use system model to analyze performance of different system configurations and various workload scenarios
  • Select the alternative with the best cost/performance relationship that satisfies the required service levels

Capacity Planning and Workload Management

capacity plan tool kit
Capacity Plan Tool Kit

The following slides (55 – 105) are provided to the student as anaid in addressing capacity planning assignments.

Capacity Planning and Workload Management

building the capacity plan
Building the Capacity Plan
  • Start at the application level
    • Each application
    • One-by-one
  • Identify key application components
  • Determine growth factors
    • Driven by business factors
      • Strategic plan
      • Tactical plan

Capacity Planning and Workload Management

components of an application
Components of an Application
  • Data
    • Functional  database and file requirements
      • Unique contribution to a shared resource
    • Program
  • Functionality
    • Background
    • Foreground
  • Program size
    • Memory occupancy
    • Libraries
  • Program frequency
  • Program type
  • Volatility

Capacity Planning and Workload Management

estimating data size 1
Estimating Data Size (1)
  • Critical component of the capacity plan
  • Very difficult to estimate early on
  • Evolutionary process
  • Key questions
    • What data does the application require?
    • What data is specific to this application?
      • What data already exists in the enterprise data store?
      • What data must be added?
    • Does the introduction of this application change the amount of data that is already captured in the data store?
    • How will the data be used?
    • How will the data be deployed?

Capacity Planning and Workload Management

estimating data size 2
Estimating Data Size (2)
  • Rule of Experience – the data must be understood before addressing implementation specifics (such as a database or file structure)
  • Accuracy is important
    • Actual amounts will evolve
    • Data sizing is iterative
  • Defined process is necessary
    • What are the appropriate questions?

Capacity Planning and Workload Management

estimating data size 3
Estimating Data Size (3)

Problem

With reference to the entity-relationship

diagram on the left, prepare an estimate

of the amount of data given the following

information:1. All current and former employees

are included.

2. The company has been in existence

for twenty-five years, averaging 400

employees per year.

3. An employee has an average of six

dependents, four addresses.

and five contacts (4 phones plus

an email).

4. There are ten salary ranges.

5. There are fifty job codes

Describe the process used as well as the

results.

Capacity Planning and Workload Management

estimating data size 4
Estimating Data Size (4)

Capacity Planning and Workload Management

estimating program requirements 1
Estimating Program Requirements (1)
  • Essential resources a program will require in order to execute
    • Memory
    • Transactions
    • Storage
    • Data Operations
  • Computer Program – a stand-alone set of instructions that accept input, process the input, and produce results

Capacity Planning and Workload Management

estimating program requirements 2
Estimating Program Requirements (2)
  • Input considerations
    • Type of input
      • Stimulus
      • Data
    • Source of the input
      • Interactive
      • Message
      • File
      • Database
    • Size of the input – how much data
    • Number of inputs

Capacity Planning and Workload Management

estimating program requirements 3
Estimating Program Requirements (3)
  • Output considerations
    • Type of output
      • Report
        • Hardcopy
        • Softcopy
      • Message
        • User-oriented
        • Process-oriented
    • Destination of the output
    • Size of the output
    • Number of outputs

Capacity Planning and Workload Management

estimating program requirements 4
Program structure

Algorithms

Subroutines

Functions

I/O operations

Files

Database

Volatility

Storage

Working data

Sort data

Input/Output data

Arrays

Frequency

How many

How often

Estimating Program Requirements (4)

Capacity Planning and Workload Management

knowing how much work is to be done
Knowing How Much Work is to be Done
  • Based on business processes
  • Mapped to software transactions
  • Key questions
    • What are the business processes?
    • What software transactions required to execute a business process?
    • How many
      • Business processes?
      • Software transactions?
    • What physical resources are required?
    • When do these processes occur?

Capacity Planning and Workload Management

what is a business process
What is a Business Process?
  • Business process
    • Set of steps that must be executed to conduct on aspect of business
      • Set of actions taken upon a business object
        • Business rules
      • Described in USE CASE
    • Conducted by the execution of one or more software transactions
      • Each software transaction has physical requirements
        • Depends on the specific nature of the transaction

Capacity Planning and Workload Management

what is a software transaction
What is a Software Transaction?
  • Computer-based program that provides some or all of the functionality required to conduct a business process
    • User interfaces
    • Inter-process interfaces
    • Input/output operations
    • Algorithm processing
      • Logic representing business rules
    • Network topology
      • Level of processing distribution
      • Inter-nodal messaging

Capacity Planning and Workload Management

elements of a software transaction
Elements of a Software Transaction
  • User interfaces
    • HCI or GUI interactions by user
      • Number submitted (by user)
      • Number transmitted (by software)
  • Inter-process interfaces
    • Direct invocations (one program to another)
    • Number of invocations executed
    • Number of times invoked (by another program)
  • Input/output interfaces
    • File reads and/or writes (media independent)
    • Database reads and/or writes
      • Indexed or non-indexed
    • Reports generated (print or screen)
  • Algorithm processing
    • Number of machine instructions required to execute a business rule
  • Network topology
    • Number of messages
    • Size of messages
    • Distribution array

Capacity Planning and Workload Management

estimating business transactions
Estimating Business Transactions

Software

Transactions

Quantity

Nodes in Topology

Transaction Type

Business Process

Capacity Planning and Workload Management

estimating software transactions
Estimating Software Transactions

SOURCE LANGUAGE

INSTRUCTIONS

SOFTWARE

TRANSACTION

SCREEN

PRINTER

#

RECORDS

#

CHAR’S

#

TABLES

#

INDEXES

#

CHAR’S

#

CHAR’S

#

RECORDS

SCREEN

NAME

# ROWS

# FILES

# FILES

Capacity Planning and Workload Management

estimating message traffic
Estimating Message Traffic

NUMBER OF

MESSAGES

MESSAGE NAME

OR IDENTIFIER

MESSAGE SIZE

Capacity Planning and Workload Management

estimating frequency
Estimating Frequency
  • When will the program execute?
    • Timelines
  • Frequency distributions
    • Nominal
    • Maximum
    • Minimum

Capacity Planning and Workload Management

timeline considerations
Timeline Considerations
  • When will the activity occur?

Expected Transaction

Volume

Transaction Volume Axis

0000

0600

1200

1800

2400

Daily Transaction Volume Distribution

Capacity Planning and Workload Management

sizing the data store 1
Stored data

Database

Distribution

Mirroring

Files

Data

Back up

Libraries

Working data

Sort

Temporary storage

Input/output buffering

Variables

Media considerations

Disk

Tape

Other

Sizing the Data Store (1)

Capacity Planning and Workload Management

sizing the data store 2
Sizing the Data Store (2)
  • Functional view
  • Technical View
    • Structure of the data store
    • Requirements of the data store manager

Capacity Planning and Workload Management

sizing the data store 3
Sizing the Data Store (3)
  • Files – physical storage structure for data
    • Governed by the rules of the file management system
    • Governed by the type of file
      • Sequential
      • Indexed
  • Databases – logical storage structure for data
    • Databases exist within files
    • Governed by the rules of the database management system
      • Data
      • Indexes
      • DBMS considerations

Capacity Planning and Workload Management

files and databases

File

Database – provides the “template” thatdescribes the organization within the file

Files and Databases

Databases exist within files

Capacity Planning and Workload Management

sizing the data store 4
Sizing the Data Store (4)
  • Database artifacts are generally sized first
    • Tables
    • Indexes
    • Databases
    • Tablespaces
  • Files not associated with the database can be sized at any time
    • Reflect the file structure

Capacity Planning and Workload Management

sizing the data store 5 the database
Sizing the Data Store (5)The Database
  • Determine the amount of application data

Then,

  • Complete this table for each row.
  • One table is required for each table in the database
  • Complete this table for each database table

Capacity Planning and Workload Management

sizing the data store 6 the database
Sizing the Data Store (6)The Database
  • Calculate the physical space requirements for data on a table-by-table basis
    • Number of rows in the table
    • Median length of a row
    • Block size defined to the DBMS
    • Overflow percentage
  • Using the process
    • Determine the number of rows that can be stored in one block
    • Determine the number of blocks that are required
    • Determine the total space requirement
    • Include an estimation safety factor

Capacity Planning and Workload Management

sizing the data store 7 the database
Sizing the Data Store (7)The Database
  • Number of rows that can be stored in one block (nr)nr = ((blocksize – (header + footer)) * overhead)/median row size
  • Number of blocks required (nb)nb = total number of rows/nr
  • Amount of disk space required (dsp)dsp = nb * blocksize
  • Estimating factor is an increasing multiplier based on the estimator’s presumption of accuracy

Capacity Planning and Workload Management

sizing the data store 8 the database
Sizing the Data Store (8)The Database
  • Index sizes must now be calculated for each index declared via a CREATE INDEX statement
    • Primary key size = nr * (bytespk + bytespa)
      • Number of estimated rows in a table
      • Number of bytes in the primary key
      • Number of bytes in the pointer address
    • (Each) search index = nr * (bytessk + bytespa)
      • Estimated number of rows
      • Number of bytes in the index key
      • Number of bytes in the pointer address

Capacity Planning and Workload Management

sizing the data store 9 the database
Sizing the Data Store (9)The Database

Assignment

You are to size this database usingthe preceding materials. Assumethat EMPLOYEE will have a searchindex on JOB_CODE and DATE_OF_HIREand that CONTACT will have a search keyof AREA_CODE. ADDRESSwill have a search key of ZIP_CODE.

Explain what considerations must be made to accommodate foreign keys

Capacity Planning and Workload Management

sizing the data store 10 non database files
Sizing the Data Store (10)Non-Database Files
  • Ordinary filesf = ((m +8) * n) * e
    • Median size of a data record in bytes
    • Number of records estimated
    • 8 bytes* per record (file overhead)
    • Estimating accuracy factor
  • Index filesf = (((m +8) * n) + i+ p) * e
    • Median size of a data record in bytes
    • Number of records estimated
    • Estimating accuracy factor
    • Size of the index
      • Number of bytes in the index
      • Size of the address pointer

* Approximate – refer to the appropriate Operating System documentation

Capacity Planning and Workload Management

sizing the data store 11 non database files
Sizing the Data Store (11)Non-Database Files

Assignment

In unloading this database for archivingpurposes, a sequential file will be created.The created file will contain a completeset of data about employees, includingall address and contact information. Thefile will also contain a complete set ofdata about dependents, including alladdress and contact information as wellas some form of connectivity to theemployee. You are to:

1. Describe each record that will be created.2. Size the sequential file.3. Explain the process that was followed.

Capacity Planning and Workload Management

sizing the application 1 considerations
Sizing the Application (1)Considerations
  • Functions performed
    • Background
    • Foreground
  • Data stores accessed
    • Files
    • Databases
  • Input/Output operations performed
    • Read/Write
    • Logical/Physical
  • Algorithms
  • Complexity
  • Frequency
  • Concurrency

Capacity Planning and Workload Management

sizing the application 2
Sizing the Application (2)
  • Generally separate by background versus foreground
  • Key measures
    • Transactions
    • Input/Output operations
    • Complexity (of algorithms)

Capacity Planning and Workload Management

sizing the application 3 definitions
Sizing the Application (3)Definitions
  • Transaction
    • Set of instructions needed to execute one instance of a business process
    • Work performed by a computer to (successfully) execute these (set of) instructions
  • Input
    • Physical – the transfer of one physical unit of data (usually referred to as a block) to memory from storage
    • Logical – the transfer of one logical unit of data (such as a record in a file or a row in a database table) to memory from storage
  • Output
    • physical – the transfer of one physical unit of data from memory to a storage medium
      • Magnetic
      • Optical
      • Electronic
      • Paper

Capacity Planning and Workload Management

sizing the application 4
Sizing the Application (4)

Capacity Planning and Workload Management

sizing the application 5
Sizing the Application (5)
  • Data is a principal driver
  • Good design is essential
  • Use optimizers to minimize executing instructions
  • Apply complexity factors
  • Size by instance

Capacity Planning and Workload Management

considerations for topology
Considerations for Topology
  • How many nodes?
  • What functions at each node?
  • What data at each node?
  • How many functions at each node?
  • How much network traffic between nodes?
    • How many messages sent?
    • How many messages received?

Capacity Planning and Workload Management

implications of topology

Y

Single Node Activity – a 2-Demensional Problem

Frequency

X

Time

Z

Frequency

Time

Y

Node

X

Multiple Node Activity – a 3-Demensional Problem

0000

0600

1200

1800

2400

Implications of Topology

NODE 5

NODE 4

NODE 3

NODE 2

NODE 1

Capacity Planning and Workload Management

implications of topology1
Implications of Topology
  • Nodal volumes not necessarily identical
  • Critical considerations
    • How many transactions?
    • In which time frame?
    • How many messages?
    • How many users?
  • Requires two views
    • Nodal view
    • System view

Capacity Planning and Workload Management

implications of topology2
Implications of Topology
  • Assignment
  • For the topology illustrated in the diagram to the left, develop a process and the associated set of tools that you would use to establish the resource requirements for a new application that is currently in development.
  • Demonstrate the viability of your approachgiven the following constraints:
    • Number of users – Baltimore (100), Ft Wayne (50), Pittsburgh (75), Kansas City (40), Missoula (30), and Denver (200)
    • Each user initiates an average of 1 transaction every two minutes
    • Each site works a standard 8-hour day from 8 AM to 5 PM (local time) Monday - Friday
    • The database is replicated and consolidated each workday evening (in Denver)
    • Each local database is backed up each weekday evening
    • Users will send one message/per hour to each of the other nodes and two messages per hour within their own node
    • A transaction accesses the database and four 80-byte fixed length files. There are no sorts.
  • Make any assumptions needed to complete the task.

Capacity Planning and Workload Management

the meaning of work
The Meaning of Work
  • What is work?
    • Accomplishing a defined task
  • How is work defined?
    • The transfer of one byte of data between a processor and secondary storage
  • What is meant by the term “work in a computer”?
    • Input operations performed
    • Instructions executed
    • Output operations performed
  • How is work measured?
    • Memory occupancy
    • Instructions executed per unit of time
  • What about processor speed?
    • Describes how fast work can be accomplished

Capacity Planning and Workload Management

work over time
Work Over Time
  • When is the work done?
  • Peaks and valleys
    • When
    • Where
  • Significance of 7 x 24 x 365
    • Does the hour matter?
    • Does the day matter?
    • Does the month matter?
    • Does the season matter?

Capacity Planning and Workload Management

profiling the requirements of time
Profiling the Requirements of Time
  • Time increments are application and installation dependent
  • Relevant time intervals
    • Hour
    • Day
    • Week
    • Month
    • Year
  • “As Required” always possible

There is a real possibility of ann-dimensional problem with regard to time

Capacity Planning and Workload Management

what must be considered 1
What Must Be Considered (1)
  • Resource requirements of individual applications
    • Over time
    • Over the topology
  • Shared resource requirements
    • What are shared resources?
    • What are the requirements for their use>
      • Over time
      • Over the topology

Capacity Planning and Workload Management

what must be considered 2
What Must Be Considered (2)

FACTOR

RESOURCE

Capacity Planning and Workload Management

capacity planning
Capacity Planning
  • The science of determining how much resource will be required over a planning horizon
  • Considerations
    • Critical resources
    • Machine capabilities
    • Workload characteristics
      • Normal workload
      • Peaks and valleys
    • Change over time
      • Growth
      • Reduction
    • System factors

Capacity Planning and Workload Management

capacity planning considerations 1
Capacity Planning Considerations (1)
  • Normal load – the expected amount of resource to be used under usual circumstances
  • Peak load – the maximum amount of resource to be used under exceptional circumstances

Capacity Planning and Workload Management

capacity planning considerations 2
Capacity Planning Considerations (2)

100%

Contingency to accommodate spikes,priorities, and/or contention

Contingency

Value

Available for Normal Use

0%

Capacity Planning and Workload Management

capacity planning considerations 3
Capacity Planning Considerations (3)
  • What do you need now?
  • What do you have?
  • What will you need at each increment in the planning horizon?
  • What is the plan?
  • What are the risks
    • Unanticipated demand
    • Unexpected surplus
    • Change in the business
  • The importance of being earnest
    • Right-sizing the environment

Capacity Planning and Workload Management

developing a capacity plan exercise
Developing a Capacity PlanExercise
  • As the systems engineering team for a mid-sized, high growth manufacturing company with an international presence, identify the source materials you will require in order to develop the enterprise’s capacity plan over a nominal business strategic planning horizon.
  • Describe the process that you will employ to develop the plan.
  • Describe the process that you will employ to keep the plan current by updating it on an annual basis.
  • Describe how your role differs from the role of the application owner.

Capacity Planning and Workload Management

ad