Detailed modeling and terminating statistical analysis
Download
1 / 88

Detailed Modeling and Terminating Statistical Analysis - PowerPoint PPT Presentation


  • 271 Views
  • Updated On :

Detailed Modeling and Terminating Statistical Analysis. Chapter 5. What We’ll Do . Explore lower-level modeling constructs Model 5-1: A generic call-center system Nonstationary arrival process Balking, three-way decisions, sets, variables, expressions, submodels, and costing Debugging

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Detailed Modeling and Terminating Statistical Analysis' - dolf


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Detailed modeling and terminating statistical analysis l.jpg

Detailed Modeling and Terminating Statistical Analysis

Chapter 5

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


What we ll do l.jpg
What We’ll Do ...

  • Explore lower-level modeling constructs

  • Model 5-1: A generic call-center system

    • Nonstationary arrival process

    • Balking, three-way decisions, sets, variables, expressions, submodels, and costing

  • Debugging

  • Model 5-2: Animating the call center model

    • Plots, global pictures, and storages

  • Model 5-3: The model with overall performance measures

    • Run conditions, model size and speed, overall performance measures

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


What we ll do cont d l.jpg
What We’ll Do ... (cont’d.)

  • Statistical analysis of simulation output (terminating systems)

    • Time frame of simulations

    • Strategy for data collection and analysis

    • Confidence intervals

    • Comparing two alternatives

    • Comparing many alternatives via the Arena Process Analyzer (PAN)

    • Searching for and optimal alternative with OptQuest

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Generic call center model 5 1 l.jpg
Generic Call Center(Model 5-1)

  • Single telephone number, 26 trunk lines

    • If all 26 lines busy, caller gets busy signal and goes away

  • Answered call gets recording asking …

    • Technical support? (76% of callers choose this)

    • Sales information? (16%)

    • Order-status inquiry? (8%)

  • Time for caller to choose ~ UNIF (0.1, 0.6)

  • All times are in minutes in this model

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Technical support calls l.jpg
Technical Support Calls

  • Get second recording asking …

    • Product type 1? (25% of tech support callers choose this)

    • Product type 2? (34%)

    • Product type 3? (41%)

  • Recording and choosing takes UNIF(0.1, 0.5)

  • If a qualified tech-support person is available for chosen product, call routed for immediate service

  • If not, call placed in (electronic) queue, subjected to annoying rock music

  • All tech support conversations ~ TRIA (3, 6, 18)

  • When call done, customer exits system

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Technical support calls cont d l.jpg
Technical Support Calls (cont’d.)

  • 4% of tech support calls need further assistance after completion of their call

    • Questions forwarded to another tech group that prepares a response; time to prepare this response ~ EXPO (60)

    • Response sent back to the same tech-support person who took the original call

    • This person calls the customer back and talks, which lasts TRIA (2, 4, 9)

    • These calls require one of the 26 trunk lines and take priority over incoming calls

    • If return call not completed on same day, it’s carried over to the next day

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Sales calls l.jpg
Sales Calls

  • Call automatically routed to sales staff

  • Sales staff is separate from tech-support staff

  • If a sales-staff person is available, call gets immediate service

  • If not, call placed in (electronic) queue, treated to soothing new-age space music

  • All sales conversations ~ TRIA (4, 15, 45)

  • When call done, customer exits system

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Order status calls l.jpg
Order-Status Calls

  • Automatically handled by phone system — no people

  • No limit on number handled at a time (but still limited by the 26 trunk lines)

  • Time for “conversation” ~ TRIA (2, 3, 4)

  • After call, 15% take option to talk to a real person (the rest exit the system)

    • These calls are routed to sales staff

    • Have same priority as incoming sales calls

    • Conversation durations ~ TRIA (3, 5, 10)

    • Then exit the system

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Call arrivals l.jpg
Call Arrivals

  • Calls accepted from 8 AM until 6 PM

  • Some staff available until 7 PM

    • Incoming calls shut out after 6 PM

    • But all calls that entered before 6 PM are answered

  • Call arrival rate varies substantially over the day

    • Data on rate (calls per hour) for each half-hour period:

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Staffing l.jpg
Staffing

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Technical support staff schedules l.jpg
Technical-Support Staff Schedules

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


New modeling issues l.jpg
New Modeling Issues

  • This is a service (not manufacturing) system

    • But can use same modeling capabilities

  • Nonstationary arrival process

    • Arrivals occur one at a time and are independent of one another

    • Average rate varies over time (would be constant for a stationary Poisson process)

    • Built into Create module (beware of popular-but-wrong methods … details in book)

  • Balking

    • Required because there are only 26 trunk lines

    • Entity arrives at queue, which is full (capacity is 0 here)

    • Entity departs from system – count these

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


New modeling issues cont d l.jpg
New Modeling Issues (cont’d.)

  • Three-way decisions

    • Entity or call can go to one of three places in model based on call type

      • Similarly, tech-support calls can go to one of three places based on product type

    • Capability available in Decide module

  • Sets

    • Groups of similar objects

    • Can be referenced by a common set name and index (1, 2, 3, …) into the set

      • Can also be referenced by original name, independent of set

    • Technical-support staff requires sets

      • An object can be a member of more than one set

    • Sets data module

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


New modeling issues cont d14 l.jpg
New Modeling Issues (cont’d.)

  • Variables and Expressions

    • Can be referenced in model by name

    • Can be one- or two-dimensional arrays, indexed by one or two integers

    • User-defined Variables

      • Store some numerical value (not a formula)

      • Can be initialized in Variable data module

      • Can be used, reassigned during the simulation run by any entity

    • User-defined Expressions

      • A name defined by a mathematical expression

      • This name can be references anywhere in the model

      • Can use constants, Variables, Attributes, system state variables, values from distribution – connected via mathematical operations

        • Can use Expression Builder to help define

      • Defined in Expression data module (Advanced Process panel)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


New modeling issues cont d15 l.jpg
New Modeling Issues (cont’d.)

  • Submodels

    • Partition simulation model into several smaller submodels

      • Can link them together, more manageable pieces

    • Just like a normal model view within a submodel

    • Submodels can also contain further submodels, etc. – hierarchical structure

    • Submodels can be externally connected to other modules or submodels

    • Navigate panel in Project Bar shows submodels, under Top-Level Model

  • Costing

    • Automatic time and cost information for entities

      • Wait, value-added, non-value-added, transfer, other

    • You must enter cost information – Entity and Resource data modules

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Key output performance measures l.jpg
Key Output Performance Measures

  • Count balks — no. of attempted incoming calls sent away due to all 26 trunk lines being busy

    • Will not model reneging — customers in queue leaving the system if they get sick of waiting

  • Total time in system, by customer type

  • Time waiting for a real person, by customer type

  • Contact time, by customer type

  • Number of calls waiting, by customer type

  • Personnel utilization

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Animation requirements l.jpg
Animation Requirements

  • No entity movement to animate here

  • Can still display queues

  • To see how well staffing matches up with load, craft appropriate plots vs. time

    • Number of calls balked

    • Lengths of queues

    • Number of idle staff

  • Strategy to improve performance — alter the staffing schedule, see if it produces a better matchup of the plots

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


System or simulation type l.jpg
System or Simulation Type

  • Terminating

    • Known starting and stopping conditions – part of model

    • Time frame is known (and finite)

  • Steady-State

    • Initial conditions are not always well defined

    • No defined stopping condition (theoretically infinite)

    • Interested in system response over the long run

  • Call-center model

    • Start at 8 AM and end at 7 PM

      • Some Technical support calls are held over, but not many – we’ll ignore this aspect (sort of … fixed below)

    • Treat the system as terminating (sort of … see below)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Arena modeling panels l.jpg
Arena Modeling Panels

  • Basic Process panel

    • Highest level of modeling

  • Advanced Process panel

    • More detailed (and different) modeling capabilities

  • Advanced Transfer panel

    • Material-handling, entity-movement capabilities

  • Blocks, Elements panels

    • Lowest level of modeling capabilities – the underlying SIMAN simulation language itself

    • Other panels are created using modules from these panels

    • Occasionally needed, but not very often

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Building the model l.jpg
Building the Model

  • Defining the Data

  • Submodel Creation

  • Divide model in sections or submodels

    • Increment the Time Period

    • Create Arrivals and Direct to Service

    • Technical Support Calls

    • Technical Support Returned Calls

    • Sales Calls

    • Order-Status calls

  • We’ll discuss each of these in turn …

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Simulation replication data l.jpg
Simulation Replication Data

  • Project replication parameters

    • Run/Setup dialog – Replication Parameters tab

    • 10 Replications of 11 hours each

    • Four options for Initialization Between Replications:

      • Initialize system (yes), initialize statistics (yes)

        • 10 independent and identical replications – no calls carried over

        • Reports for each day separately

      • Initialize system (yes), initialize statistics (no)

        • 10 independent and identical replications – no calls carried over

        • Cumulative summary reports (day 1, days 1-2, days 1-3, …, days 1-10)

      • Initialize system (no), initialize statistics (yes): Selected

        • 10 continuous days – calls carried over

        • Reports are by replication (day)

      • Initialize system (no), initialize statistics (no)

        • 10 continuous days – calls carried over

        • Cumulative summary reports

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Schedule data l.jpg
Schedule Data

  • Schedules

    • Enter into Schedule data module

    • 13 schedules required

      • One for each of the 11 technical-support people

      • One for the sales staff overall

      • The arrival process (Type = Arrival, not Capacity)

    • Use Graphical schedule editor (initially)

    • Use Edit via Dialog (or Edit via Spreadsheet) if you need trailing zeros in the capacity to fill out the cycling time window

      • We need this in this model due to not Initializing System between replications … see book for details

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Resource data l.jpg
Resource Data

  • Define resources

    • Use Resource data module

    • 13 resources

      • Sales staff

      • 11 technical support staff

      • Trunk Line (single resource with 26 units)

    • Enter Schedule Name for all but Trunk Line

      • For resources on a Schedule, use Ignore option for Schedule Rule to ensure correct cross-day modeling … details in book

    • Add hourly wage under costing data

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Sets data l.jpg
Sets Data

  • Use Set data module (Basic Process panel)

  • Develop three Resource sets for technical support staff

    • Product 1

      • Charity, Noah, Molly, Anna, Sammy

    • Product 2

      • Tierney, Sean, Emma, Anna, Sammy

    • Product 3

      • Shelley, Jenny, Christie, Molly, Anna, Sammy

    • Note that Anna and Sammy are in all three sets

    • Consistently listed the more versatile staff at the end of the list in each set … “save” them … discussed later

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Sets data cont d l.jpg
Sets Data (cont’d.)

  • Develop two Tally sets

    • Tech Calls

      • Product 1 Call, Product 2 Call, Product 3 Call

    • Returned Time

      • Return 1 Call, Return 2 Call, Return 3 Call

    • Sets used to collect statistics by product type

  • Develop a Counter set

    • Keep track of number of balks per half-hour period

    • 22 counters – one for each half-hour period

    • First defined 22 counters in Statistic data module (Advanced Process panel)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Variables and expressions data l.jpg
Variables and Expressions Data

  • Variables

    • Use Variable data module to define thee variables

      • Period (the current time period)

      • Busy Per Period (busy signals in current time period)

      • Per Period Balk (total balks for last completed time period)

      • Note – explicit use of Variables module is required only if you want a Variable to have a non-zero initial value

  • Expressions

    • Use Expression data module to define three expressions

      • Returned Tech Time, for duration of returned tech-support calls: TRIA(2, 4, 9)

      • Tech Time, for duration of tech-support calls : TRIA(3, 6, 18)

      • Available 1, Available 2, and Available 3

        • Sum of currently available, but idle, resources by product type, for staffing plots

        • Use Expression Builder … details in book, model

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel creation l.jpg
Submodel Creation

  • Object/Submodel/Add Submodel menu option to create a submodel … we’ll use six submodels

    • Define (right-click, then Properties)

      • Name

      • Number of entry, exit points (could be 0 if there’s no flow interaction)

    • Move between submodels: Navigate panel, Named Views, or mouse

      • Double-click on a submodel to open it

      • When in a submodel, right-click in an empty place, then Close Submodel, to go up

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Time period counter submodel l.jpg
Time Period Counter Submodel

  • Increments the time period counter

  • No entry or exit points – interacts via Variables, not flow

  • Create Counter Entity – Create module

    • Time Between Arrivals – 660 minutes (constant)

  • Assign Period – Assign module

    • Initialize Period variable value to zero

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Time period counter submodel cont d l.jpg
Time Period Counter Submodel (cont’d.)

  • Assign Variables – Assign module

    • Increment Period variable for the next half-hour period

    • Assign Per Period Balk to Busy Per Period variable value (number of calls balked during previous half hour)

    • Set Busy Per Period variable to zero to start balk counting during the half hour starting now

  • Check Period – Decide module

    • 2-Way by Condition

    • Determine if there are still more periods in this day (i.e., if Period < 22)

      • Yes: Delay for a half hour – Delay module, then loop back

      • No: Dispose of entity – Dispose module

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Create and direct arrivals submodel l.jpg
Create and Direct Arrivals Submodel

  • Creates arrivals, checks for available trunk line, and directs to appropriate type of service

  • No entry points

  • Three exit points

    • Tech Call, Sales Call, Order Status Call

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel logic l.jpg
Submodel Logic

  • Create arriving calls

    • If a trunk line is available – seize one

      • Assign Arrival Time attribute (for use downstream)

      • Delay to listen to recording

      • Determine call type

      • Direct call and assign entity type

    • Else (all trunk lines are busy)

      • Count balked call

      • Increment Busy Per Period counter

      • Dispose of call

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Develop submodel l.jpg
Develop Submodel

  • Create arriving calls – Create module

    • Time Between Arrivals

      • Type: Schedule

      • Schedule Name: Arrival Schedule

        • Was defined when we defined the data for the model

  • Check for available trunk line

    • Queue/Seize module combination (Blocks panel)

    • Set queue capacity to zero

      • If trunk line available, resource seized in following Seize module

      • If no truck line available, entity will automatically balk

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Develop submodel cont d l.jpg
Develop Submodel (cont’d.)

  • Assign arrival time – Assign module

    • Use Arena variable TNOW = current simulation clock

  • Delay for Recording – Delay module

    • Used Delay module from Blocks panel

    • Be careful of units – no choice here (uses Base Time Units)

  • Direct call – Decide module

    • Use N-way by Chance option

    • Enter probabilities as percents (0 – 100)

  • Assign call type – Assign module

    • Assign entity type to call type

    • Three modules

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Develop submodel cont d34 l.jpg
Develop Submodel (cont’d.)

  • Balking entities

    • Count balked call – Record module

      • Record into counter set Busy Lines

        • Set index is the variable Period

    • Increment Busy Per Period variable – Assign module

    • Dispose of balked call call – Dispose module

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Technical support calls submodel l.jpg
Technical Support Calls Submodel

  • Logic for servicing technical support calls

  • One entry point

  • One exit point – follow-up calls

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel logic36 l.jpg
Submodel Logic

  • Delay to listen to recording – Delay module

    • UNIF(0.1, 0.5) minutes

  • Determine product type – Decide module

    • N-way by Chance

  • Seize technical support person

    • Seize module – Advanced Process panel

    • Request from appropriate set for product type

    • Preferred order within the set

      • Save more versatile employees for other things

    • Save set index (particular tech-support person) in attribute Tech Agent Index

      • In case returned tech call is needed – get same tech-support person to call customer back

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel logic cont d l.jpg
Submodel Logic (cont’d.)

  • Save product type and call start time – Assign module

    • Save type (1, 2, or 3) in attribute Product Type

    • Assign value from TNOW to attribute Call Start

  • Delay for call – Delay module

    • Use value from expression Tech Time

  • Release tech-support person and trunk line

    • Release module – Advanced Process panel

    • Use set index in attribute Tech Agent Index to release the particular tech-support person assigned from set

    • Release the seized unit of Trunk Line resource

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel logic cont d38 l.jpg
Submodel Logic (cont’d.)

  • Record call and line time – Record module

    • Time Interval type

    • Tally set Tech Calls

    • Set index Product Type

    • Records only the time spent during the tech-support conversation (necessary?)

  • Record tech line time – Record module

    • Time Interval type

    • Tally Tech Support Line Time (not a Tally set)

    • Use Arrival Time attribute set when call first arrived, so this records the total time in the system so far

    • Direct to exit point

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Returned tech calls submodel l.jpg
Returned Tech Calls Submodel

  • Logic for returned tech calls

  • One entry point, no exit points

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel logic40 l.jpg
Submodel Logic

  • Check for returned call – Decide module

  • If no returned call is needed

    • Dispose of entity

  • If a returned call is needed

    • Entity Type set to Returned Call – Assign module

    • Delay for response time – Delay module

    • Direct by product type

      • N-way by Condition based on attribute Product Type

    • Seize tech-support person and trunk line

      • Seize module: Seize specific member of appropriate Resource set

        • Use Set Index Tech Agent Index attribute

      • Seize Trunk Line

Note use of “==” to check for equality.

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel logic cont d41 l.jpg
Submodel Logic (cont’d.)

  • Delay for call time – Delay module

    • Expression Returned Tech Time

  • Release tech person and trunk line – Release module

  • Record returned time – Record module

    • Use beginning-time attribute Arrival Time, defined when call first arrived, so this records total time in system

    • Use Tally Set Return Time indexed by Product Type

  • Dispose of entity – Dispose module

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Sales calls submodel l.jpg
Sales Calls Submodel

  • Logic for sales calls

  • One entry point, no exit points

  • Uses a Shared Queue

    • Single queue

    • Shared by two or more seize activities

      • In this case, the “real” incoming sales calls, as well as those order-status calls requiring more than just the automated response

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel logic43 l.jpg
Submodel Logic

  • Seize sales person – Seize module

    • Shared queue declared in Queue data module

  • Delay for call – Delay module

  • Release sales person and Trunk Line – Release module

  • Record call time – Record module

    • Records elapsed time from call’s original arrival until now

  • Dispose of entity - Dispose module

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Order status calls submodel l.jpg
Order-Status Calls Submodel

  • Logic for order-status calls

  • One entry point, no exit point

  • Shared queue used when sales person required

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Submodel logic45 l.jpg
Submodel Logic

  • Delay for call – Delay module

  • Decide if Sales person required – Decide module

  • If sales person is required

    • Seize sales person – Seize module, shared queue

    • Follow-up delay – Delay module

    • Release sales person – Release module

  • Record call – Record module

    • Elapsed time from call’s arrival to system up to now

  • Release trunk line – Release Module

  • Dispose of entity – Dispose module

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Finding and fixing model errors l.jpg
Finding and Fixing Model Errors

  • Arena picks up “simple” errors in Check phase, and leads you to them via Find and Edit buttons in Errors/Warnings windows

    • Undefined variables, attributes, resources

    • Unconnected modules

    • Duplicate module names

    • Typos

  • Other kinds of errors are more complex, can’t be detected without trying to run — options on Run Interaction toolbar or on Run menu

  • Only mention capabilities here; see text for details

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Finding and fixing model errors cont d l.jpg
Finding and Fixing Model Errors (cont’d.)

  • Run Controller — Command-driven window to control, display details about model operation and underlying SIMAN code

  • Trace — Follow active modules, selected variables

  • Highlight active module – highlights the active module during the simulation run

  • Layers – gives control over what you see during the simulation run

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Finding and fixing model errors cont d48 l.jpg
Finding and Fixing Model Errors (cont’d.)

  • Break on Module; Break — stop run when entity hits a selected module, at a specific time, or when a selected entity is about to become active

  • Watch — select expressions to display in a window as model runs

  • Look at reports when model is running or paused

    • Remember to close reports windows

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Model 5 2 animating the model l.jpg
Model 5-2: Animating the Model

  • No “normal” entity animation — just plots, queues, a few other “data” animations

  • Plots (all vs. time on horizontal axis)

    • Queue lengths (as in earlier models)

    • Balks per period — reason for variable Per Period Balk

    • Number of tech support people available for each product type — reason for the “Available” expressions defined in Expressions module

    • With multiple plots, configure first one, then copy/edit for others to get consistent look and feel; snap to grid to align

  • Variable animations for Period and Day

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Animating the model cont d l.jpg
Animating the Model (cont’d.)

  • Created digital clock “by hand” (details in text)

    • Why not ready-made animated clocks? We didn’t reset the system state between replications, so internal clock just keeps increasing

  • Resource and queue animations

  • Just for realism — doesn’t add any analysis value

  • Resource button from Animate toolbar

    • Take pictures from libraries (.plb files), different states

  • Queue button from Animate toolbar

  • Add various text annotations, boxes, etc.

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Animating the model cont d51 l.jpg
Animating the Model (cont’d.)

  • Storages for further animation

    • Requires using Delay modules from Blocks panel, not Advanced Process panel

      • We did this only for the delay listening to the first recorded message

    • Enter a Storage name in the Blocks Delay module (Message 1)

    • Storage button from Animate Transfer toolbar

  • Animated variables

    • Number of available trunk lines

    • Number of available salespeople

    • Number of sales calls in progress

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Model 5 3 model for analysis with overall performance measure l.jpg
Model 5-3: Model for Analysis with Overall Performance Measure

  • Modify the call-center model for intensive study

    • Different run conditions – to allow valid statistical analysis

    • Smaller size – to continue to fit in academic version and make room for other enhancements

    • Faster – to allow for extensive analysis

    • New overall performance measures to consider both resource costs and customer-oriented performance

  • Base on Model 5-1 rather than Model 5-2 since the latter adds only animation and we’re now crunching numbers

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Run conditions l.jpg
Run Conditions Measure

  • Want valid terminating statistical analysis

    • New replications must start independently with no model-state carry-over

  • Run/Setup/Replication Parameters

    • Check “Initialize System Between Replications”

      • Still check “Initialize Statistics Between Replications”

    • Get truly independent and identically distributed replications

    • Unreturned tech-support follow-up calls lost – unrealistic

    • Compromise – redefine a “replication” to be a five-day work week … Monday-Thursday returned tech-support calls carried over, Friday-night ones lost

      • Run/Setup/Replication Parameters: Replication Length = 5 Days, specify 11 Hours Per Day

  • Run/Run Control – Batch Run (No Animation)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Slimming down and speeding up l.jpg
Slimming Down and Speeding Up Measure

  • Academic version places several different kinds of limits on model “size”

    • Max of 150 concurrent entities … though millions could pass through … this limit is not a problem here

    • Max of 150 “module instances” … includes flowchart modules, and each entry (line) in data modules … this limit is a problem

    • Also need room to add new output performance measures

  • Reduce number of module instances

    • Eliminate many statistical accumulators … included getting rid of lines in data modules, entire flowchart modules, and unchecking stat-collection boxes … details in book

  • This also increased speed by factor of 3 to 4

    • Important since this model will be exercised intensively

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Overall performance measures l.jpg
Overall Performance Measures Measure

  • Form an overall cost measure – reduce, minimize

    • What controllable input parameters affect cost? How?

  • Two components to cost

    • Due to staffing and resources – tangible, measurable

    • Due to poor customer service – intangible, hard to measure

  • Staffing and resource costs

    • Hourly salaries: $18 for sales, $16 to $20 for tech-support depending on training (see Resource data module)

      • Salaries paid whenever person is on duty, whether busy or not

    • Get current weekly payroll of $13,110 = Staff Cost variable

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Generalize for more staff l.jpg
Generalize for More Staff Measure

  • Try to improve service via more staff

    • Will certainly increase staff cost

    • Try to improve customer service to make it worthwhile

  • Base-model results – worst staffing shortfalls are between 11:30AM and 3:30PM

  • Add sales and tech-support staff for that four-hour period (half-hours 8 through 15)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Additional sales staff l.jpg
Additional Sales Staff Measure

  • Add variable New Sales to be the number of additional salespersons to add in periods 8-15

    • Define in Variable data module

    • Use in Schedule data module (under Sales Schedule) … add to number of sales staff in base model in periods 8-15

      • Must use Edit via Dialog or Spreadsheet since Graphical Schedule Editor cannot handle Variables

  • Cost for each new salesperson: $15/hour

    • Each will work 20 hours/week, so cost $300/week

    • Variable New Sales Cost set to 300

    • Additional cost is (New Sales) * (New Sales Cost), used in Expression for for new resource cost (New Res Cost)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Additional tech support staff l.jpg
Additional Tech-Support Staff Measure

  • Possibly add new tech-support staff for products 1, 2, 3 (only), and for all products

    • Variables New Tech 1, New Tech 2, New Tech 3

      • Named Larrys, Moes, and Curlys, respectively

      • Each paid $14/hr * 20 hrs/week = $280/week (variable LMC cost)

    • Variable New Tech All

      • Named Hermanns

      • Each paid $17/hr * 20 hrs/week = $340/week (variable Her cost)

    • Resource data module – define resources Larry, Moe, Curly, Hermann … and hourly costs for them (not used)

    • Set data module – add Larry, Moe, Curly, Hermann to appropriate resource sets

    • Schedule data module – add Larry, Moe, Curly, Hermann schedules … can’t use Graphical Schedule Editor

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Changing the number of trunk lines l.jpg
Changing the Number of Trunk Lines Measure

  • Each trunk line costs a flat $89/week, including all calls (even long-distance)

  • Is 26 the right number?

  • To change it, just edit the Capacity entry in the Resource data module

  • Add variable Line Cost to be $89, multiply by number of trunk lines (MR(Trunk Line)) to get weekly cost of trunk lines

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Total new resource cost l.jpg
Total New Resource Cost Measure

  • Define an Expression called New Res Cost:

    New Sales * New Sales Cost

    + (New Tech 1 + New Tech 2 + New Tech 3) * LMC Cost

    + New Tech All * Her Cost

    + Line Cost * MR(Trunk Line)

  • Does not depend on what happens during simulation … used only at end in Statistic module

  • Does not include cost of the base-model human staff (sales, tech-support) … viewed as sunk, and constant for all variants of staffing changes

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Costs for putting customers on hold l.jpg
Costs for Putting Customers on Hold Measure

  • Impute a cost for making customers wait on hold

    • Trade off against resource costs

    • Use model to understand, improve, optimize this tradeoff

    • Such customer-dissatisfaction costs are hard to quantify

  • People have a “tolerance” for holding

    • Tech-support calls: 3 minutes (variable Tolerance Tech)

    • Sales calls: 1 min. (Tolerance Sales)

    • Order-status calls: 2 min. (Tolerance Order Status)

  • Beyond the tolerance point, system incurs cost of

    • Tech-support calls: $1.67/min. (variable TWT Cost)

    • Sales calls: $3.72/min. (SWT Cost)

    • Order-status calls: $1.58/min. (OSWT Cost)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Costs for putting customers on hold cont d l.jpg
Costs for Putting Customers on Hold Measure (cont’d.)

  • Accumulate “excess” waiting time (time past tolerance) for each call type

    • Assign module when call is done

    • Use built-in Arena attribute ENTITY.WAITTIME

      • Accumulates total of times in queues as entity goes along, and other “Wait”-allocated times … but there are none upstream in this model so this attribute will have the waiting time on hold

      • Requires that Costing box be checked in Run/Setup/Project Parameters

    • Variable Excess Tech Wait Time accumulates via adding in for each tech-support call

      MAX( ENTITY.WAITTIME - Tolerance Tech, 0 )

    • At end of run, multiply Tech Wait Time by TWT Cost

    • Similarly for Sales, Order-Status calls

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Total cost l.jpg
Total Cost Measure

  • Adding together all the costs, get the overall economic (cost) performance measure

    Total Cost = New Res Cost

    + Excess Sales Wait Time * SWT Cost

    + Excess Status Wait Time * OSWT Cost

    + Excess Tech Wait Time * TWT Cost

    + Staff Cost

  • This is defined in Statistic data module

    • Type = Output – already being computed, just report it

    • In Category Overview Report, get via User Specified  Other  Output

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Percent busy requirement l.jpg
Percent-Busy Requirement Measure

  • Above cost performance measure ignores calls balked away due to no trunk line … busy signal

    • Clearly, undesirable – very hard to put a cost on it

    • Instead, have a strong goal to limit this to no more than 5% of incoming calls … a model configuration not satisfying this will be deemed unacceptable no matter how attractive (low) the cost may be

    • Like a constraint except it’s on an output, not an input … call it a requirement

  • Compute via two Record modules in arrival submodel to count incoming and balked calls … and Percent Busy line in Statistic module

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Aside generality of models l.jpg
Aside – Generality of Models Measure

  • We could have done a lot of things in very different ways in this model

    • Using Arena’s costing functions more and doing fewer of our own external calculations

    • Reparameterize using only “primitive” parameters (e.g., hourly wage rates) and programming Arena to do the calculations

  • How much of this you do depends on model’s intended use and users

  • Tradeoff between generality (elegance?) vs. time spent building the model

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Results l.jpg
Results Measure

  • Base case (no more people, 26 trunk lines)

    • Total cost (for the week) = $34K

    • Percent busy signals = 11% (unacceptable)

  • Added one resource unit for each type

    • New Sales, New Tech 1, New Tech 2, New Tech 3, New Tech All, and go to 27 trunk lines

    • Total cost (for the week) = $28K

      • Added resources reduced customer waiting time by more than enough to cover their cost

    • Percent busy signals = 3% (acceptable)

      • Extra trunk line, plus added resources to move calls through

  • Is the modification truly and reliably better???

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Statistical analysis of output from terminating simulations l.jpg
Statistical Analysis of Output from Terminating Simulations Measure

  • Random input leads to random output (RIRO)

  • Run a simulation (once) — what does it mean?

    • Was this run “typical” or not?

    • Variability from run to run (of the same model)?

  • Need statistical analysis of output data

    • From a single model configuration

    • Compare two or more different configurations

    • Search for an optimal configuration

  • Statistical analysis of output is often ignored

    • This is a big mistake – no idea of precision of results

    • Not hard or time-consuming to do this – it just takes a little planning and thought, then some (cheap) computer time

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Time frame of simulations l.jpg
Time Frame of Simulations Measure

  • Terminating: Specific starting, stopping conditions

    • Run length will be well-defined (and finite)

  • Steady-state: Long-run (technically forever)

    • Theoretically, initial conditions don’t matter (but practically they usually do)

    • Not clear how to terminate a simulation run

  • This is really a question of intent of the study

  • Has major impact on how output analysis is done

  • Sometimes it’s not clear which is appropriate

  • Here: Terminating (steady-state in Section 6.3)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Strategy for data collection and analysis l.jpg
Strategy for Data Collection and Analysis Measure

  • For terminating case, make IID replications

    • Run/Setup/Replication Parameters: Number of Replications field

    • Check both boxes for Initialize Between Replications

  • Separate results for each replication – Category by Replication report

    • Model 5-3, base case, 10 replications

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Strategy for data collection and analysis cont d l.jpg
Strategy for Data Collection and Analysis Measure (cont’d.)

  • Category Overview report will have some statistical-analysis results of the output across the replications

  • How many replications?

    • Trial and error (now)

    • Approximate number for acceptable precision (below)

    • Sequential sampling (Chapter 11)

  • Turn off animation altogether for max speed

    • Run/Run Control/Batch Run (No Animation)

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Confidence intervals for terminating systems l.jpg
Confidence Intervals for MeasureTerminating Systems

  • Using formulas in Chapter 2, viewing the cross-replication summary outputs as the basic data:

  • Possibly most useful part – 95% confidence interval on expected values

  • This information (except standard deviation) is in Category Overview report

    • If > 1 replication specified, Arena uses cross-replication data as above

    • Other confidence levels, graphics – Output Analyzer

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Half width and number of replications l.jpg
Half Width and Number of Replications Measure

  • Prefer smaller confidence intervals — precision

  • Notation:

  • Confidence interval:

  • Half-width =

  • Can’t control t or s

  • Must increase n — how much?

Want this to be “small,” say

< h where h is prespecified

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Half width and number of replications cont d l.jpg
Half Width and Number of Replications Measure (cont’d.)

  • Set half-width = h, solve for

  • Not really solved for n (t, s depend on n)

  • Approximation:

    • Replace t by z, corresponding normal critical value

    • Pretend that current s will hold for larger samples

    • Get

  • Easier but different approximation:

s = sample standard

deviation from “initial”

number n0 of replications

n grows quadratically

as h decreases.

h0 = half width from “initial” number n0 of replications

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Interpretation of confidence intervals l.jpg
Interpretation of Confidence Intervals Measure

  • Interval with random (data-dependent) endpoints that’s supposed to have stated probability of containing, or covering, the expected valued

    • “Target” expected value is a fixed, but unknown, number

    • Expected value = average of infinite number of replications

  • Not an interval that contains, say, 95% of the data

    • That’s a prediction interval … useful too, but different

  • Usual formulas assume normally-distributed data

    • Never true in simulation

    • Might be approximately true if output is an average, rather than an extreme

    • Central limit theorem

    • Issues of robustness, coverage, precision – details in book

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Comparing two alternatives l.jpg
Comparing Two Alternatives Measure

  • Usually, want to compare alternative system configurations, layouts, scenarios, sensitivity analysis … here just two alternatives

  • Base case of Model 5-3, vs. adding one resource unit for each type

    • New Sales, New Tech 1, New Tech 2, New Tech 3, New Tech All, and go to 27 trunk lines

    • Earlier, one run of each suggested big differences … real?

  • Reasonable but not-quite-right idea: Make confidence intervals on expected outputs from each alternative, see if they overlap

    • Doesn’t allow for a precise, efficient statistical conclusion

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Compare means via the output analyzer l.jpg
Compare Means via the Output Analyzer Measure

  • Output Analyzer is a separate application that operates on .dat files produced by Arena

    • Not installed by default from book CD – need custom install

    • Launch separately from Windows, not from Arena

  • To save output values (Expressions) of entries in Statistic data module (Type = Output) – enter filename.dat in Output File column

    • Just did for Total Cost, not Percent Busy

    • Will overwrite this file name next time … either change the name here or out in Windows before the next run

    • .dat files are binary … can only be read by Output Analyzer

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Compare means via the output analyzer cont d l.jpg
Compare Means via the Output Analyzer Measure (cont’d.)

  • Start Output Analyzer, open a new data group

    • Basically, a list of .dat files of current interest

    • Can save data group for later use – .dgr file extension

    • Add button to select (Open) .dat files for the data group

  • Analyze/Compare Means menu option

    • Add data files … “A” and “B” for the two alternative

    • Select “Lumped” for Replications field

    • Title, confidence level, accept Pared-t Test, Scale Display

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Compare means via the output analyzer cont d78 l.jpg
Compare Means via the Output Analyzer Measure (cont’d.)

  • Results:

  • Confidence interval on difference misses 0, so conclude that there is a (statistically) significant difference

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Evaluating many alternatives with the process analyzer pan l.jpg
Evaluating Many Alternatives with the Process Analyzer (PAN) Measure

  • With (many) more than two alternatives to compare, two problems are

    • Simple mechanics of making the possibly many parameter changes, making the runs, keeping track of the many output files

    • Statistical methods for drawing reliable and useful conclusions

  • Process Analyzer (PAN) addresses these

  • PAN operates on program (.p) files – produced when .doe file is run (or just checked)

  • Start PAN from Arena (Tools/Process Analyzer) or via Windows

  • PAN runs on its own, separate from Arena

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Pan scenarios l.jpg
PAN Scenarios Measure

  • A scenario in PAN is a combination of:

    • A program (.p) file

    • Set of input controls that you choose

      • Chosen from Variables and Resource capacities – think ahead

      • You fill in specific numerical values

    • Set of output responses that you choose

      • Chosen from automatic Arena outputs or your own Variables

      • Values initially empty … to be filled in after run(s)

    • To create a new scenario in PAN, double-click where indicated, get Scenario Properties dialog

      • Specify Name, Tool Tip Text, .p file, controls, responses

      • Values of controls initially as in the model, but you can change them in PAN – this is the real utility of PAN

      • Can duplicate (right-click, Duplicate) scenarios, then edit for a new one

    • Think of a scenario as a row

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Pan projects and runs l.jpg
PAN Projects and Runs Measure

  • A project in PAN is a collection of scenarios

    • Program files can be the same .p file, or .p files from different model .doe files

    • Controls, responses can be the same or differ across scenarios in a project – usually will be mostly the same

    • Think of a project as a collection of scenario rows – a table

    • Can save as a PAN (.pan extension) file

  • Select scenarios in project to run (maybe all)

  • PAN runs selected models with specified controls

  • PAN fills in output-response values in table

    • Equivalent to setting up, running them all “by hand” but much easier, faster, less error-prone

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Running model 5 3 with pan l.jpg
Running Model 5-3 with PAN Measure

  • Scenarios

    • Base case (no additional resources)

    • Imagine $1200/week to spend on each additional resource type, one at a time (no mixed enhancements)

    • 7 scenarios in all (details in book)

    • Select all to run (click on left of row, Ctrl-Click or Shift-Click for more)

    • To execute, or Run/Go or F5

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Statistical comparisons with pan l.jpg
Statistical Comparisons with PAN Measure

  • Model 5-3 alternatives were made with 10 replications each

    • Better than one replication, but what about statistical validity of comparisons, selection of “the best”?

  • Select Total Cost column, Insert/Chart (or or right-click on column, then Insert Chart)

    • Chart Type: Box and Whisker

    • Next, Total Cost; Next defaults

    • Next, Identify Best Scenarios

      • Smaller is Better, Error Tolerance = 0 (not the default)

      • Show Best Scenarios; Finish

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Statistical comparisons with pan cont d l.jpg
Statistical Comparisons with PAN Measure (cont’d.)

  • Vertical boxes: 95% confidence intervals

  • Red scenarios statistically significantly better than blues

    • More precisely, red scenarios are 95% sure to contain the best one

    • Narrow down red set – more replications, or Error Tolerance > 0

    • More details in book

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Searching for an optimal alternative with optquest l.jpg

Objective function is the simulation model Measure

Contractual obligation, space limitation

Nobody’s fired

Space limitation

An output requirement, not an input constraint

Searching for an Optimal Alternative with OptQuest

  • The scenarios just considered with PAN are just 7 of many, many possibilities

  • Try to find input-control values that minimize Total Cost while keeping Percent Busy < 5%

  • Formulate as an optimization problem:

    Minimize Total Cost

    Subject to 26  Trunk Lines  50

    New Sales, New Tech 1, New Tech 2, New Tech 3, New Tech All  0

    New Sales + New Tech 1 + New Tech 2 + New Tech 3 + New Tech All  15

    Percent Busy < 5%

    • Reasonable starting place – best acceptable scenario so far: Add 3 New Tech All

    • Where to go from here? Explore all of feasible six-dimensional space exhaustively? No.

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Optquest l.jpg
OptQuest Measure

  • OptQuest searches intelligently for an optimum

    • Like PAN, OptQuest

      • Runs as a separate application … can be launched from Arena

      • “Takes over” the running of your model

      • Asks that you identify the input controls and the output (just one) response objective

    • Unlike PAN, OptQuest

      • Asks that you specify constraints on the input controls

      • Asks that you specify requirements on outputs

      • Decides itself what input-control-value combinations to try

      • Uses internal heuristic algorithms to decide how to change the input controls to move toward an optimum configuration

  • You specify stopping criterion for the search

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Using optquest l.jpg
Using OptQuest Measure

  • Tools/OptQuest for Arena

  • New session (File/New or Ctrl+N or )

    • Make sure the desired model window is active

  • Select controls – Variables, Resource levels

    • Trunk Line, New Tech 1, 2, 3, and New Tech All

    • Bounds: 26  Trunk Line  50, others between 0 and 15

    • Type is Discrete for all, Input Step Size 1

  • Constraints – enter

    New Sales + New Tech 1 + New Tech 2 + New Tech 3 + New Tech All <= 15

  • Objective and Requirement

    • Total Cost Response – Select Minimize Objective

    • Percent Busy Response – Select Requirement, enter 5 for Upper Bound

    • Reorder

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


Using optquest cont d l.jpg
Using OptQuest Measure (cont’d.)

  • Options window – computational limits, procedures

    • Time tab – accept Run for 10 minutes default

    • Precision tab – vary number of replications from 3 to 10

    • Preferences tab – various settings (accept defaults)

  • Can revisit Controls, Constraints, Objective and Requirements, or Options windows via

  • Run via wizard (first time through a new project), or Run/Start or

  • Open View/Status and Solutions and View/Performance Graph to watch progress

  • Can’t absolutely guarantee a true optimum

    • Usually finds far better configuration than possible by hand

Chapter 5 – Detailed Modeling and Terminating Statistical Analysis


ad