CE 451 - Urban Transportation Planning and Modeling
Download
1 / 98

CE 451 - Urban Transportation Planning and Modeling Iowa State University - PowerPoint PPT Presentation


  • 128 Views
  • Uploaded on

CE 451 - Urban Transportation Planning and Modeling Iowa State University Calibration, Adjustment and Validation. Sources: Calibration and Adjustment of System Planning Models Note: Date = 1990 (need to adjust for inflation, other changes)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' CE 451 - Urban Transportation Planning and Modeling Iowa State University ' - ezekiel-melton


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

CE 451 - Urban Transportation Planning and Modeling

Iowa State University

Calibration, Adjustment and Validation

Sources:

Calibration and Adjustment of System Planning Models

Note: Date = 1990 (need to adjust for inflation, other changes)

Model Validation and Reasonableness Checking Manual

NHI course on Travel Demand Forecasting (152054A)


Objectives
Objectives:

  • Identify and interpret trends affecting travel demand

  • Explain difference between calibration and validation

  • Identify critical reasonableness checks

    • socioeconomic

    • travel survey

    • network

    • trip generation

    • mode split

    • trip assignment


Terminology
Terminology

  • Model Calibration

    • Estimate parameters

    • Match observations (OD, AADT)

  • Model Validation

    • Reasonableness checks

    • Sensitivity checks

  • Special generators

  • Screen lines, cut lines, cordons

Is the model sensitive to policy options?


Planner responsibilities
Planner responsibilities

  • Actively involve all participants

    • Modelers

    • Planners

    • Decision makers

    • Public

  • Fairly present all alternatives

    • Timely

    • Unbiased

  • Identify (clearly) the decision making process

    • Who, when, and how

    • Allows input from all interested groups

  • You must rely on the TDM

    • Therefore, must be validated

    • Accurate and easy to understand (documented)

www.readysetpresent.com


How do you judge a model recommend improvement
How do you judge a model/recommend improvement?

Scrutinize these characteristics:

  • Data requirements

  • Logic of structure and conceptual appeal

  • Ease of calibration

  • Effectiveness of the model (accuracy, sensitivity)

  • Flexibility in application

  • Types of available outputs

  • Operational costs

  • Experience and successes to date

  • Public or private domain availability

cio.com


Trends affecting travel demand
Trends Affecting Travel Demand

  • Planners should monitor the following trends:

    • Demographics

    • Composition of the labor force

    • Immigration and emigration

    • Regional economic development

    • Modal shares

    • Vehicle occupancy

    • Average trip length

    • Freight transport

  • Are trends consistent with assumptions made in the modeling process?

Must be aware of trends to ensure reasonable forecasts

Image sources: scu.edu; usda.gov; illinois.edu; uwex.edu; mwcog.org; fhwa.gov; transportation1.org


How sensitive is travel to fuel price?

doe.gov

http://www.eia.doe.gov/oiaf/aeo/pdf/trend_4.pdf


Tips for building a good model
Tips for building a good model*

  • Build accurate road network

  • Use aerial photos behind

  • Make sure road attributes are correct, esp. traffic

  • Use hourly counts

  • Income and auto ownership don’t fully explain travel

  • age, gender, life cycle and personal interest come into play

  • Use survey data

    • visualize these data

    • Survey can be done cheaply

    • Cooperation will be good if there’s a good reason for it – mayor sends letter, e.g.

  • Employer based surveys get good response (but may be biased)

    • Some will give home addresses, customer addresses, license plates

  • Use trip chaining (tour based) and activity based trip generation

  • We don’t know much about attractions – ITE sample too small – do your own

  • Drive the network using GPS

  • Get some data and do some statistics to derive your parameters

*Howard Slavin, Caliper Corp. 3/13/04 peer review


Tips for building a good model1
Tips for building a good model*

  • Some models are completely made up except traffic counts

    • See if you really believe the counts

  • Create your OD matrix from ground counts

    • May be better than trip gen/dist if you “made up” the whole model (no surveys)

    • TransCAD has a tool for this

    • If still want to use trip gen/dist, this method can be used to determine K factors

    • Could also use the row and column totals as the dependent variables in your trip gen model

  • Examine individual links after model run

    • Where are the trips coming from and going to that use the link?

    • In TransCAD, what is the process used to determine this (for a particular link)?

    • In TransCAD, what is the process used to show where traffic from a particular zone is going to?

  • Familiarity with your region is helpful

*Howard Slavin, Caliper Corp. 3/13/04


Sources of error
Sources of Error

  • Coding

  • Sampling

  • Computation (if done by hand)

  • Specification

  • Data Transfer

  • Data aggregation

Improper structure of model, e.g., wrong variables


Key concepts
Key Concepts

  • Not enough attention on model evaluation and reasonableness checks

  • Checks should be performed after each step

    • reduces error propagation

Errors can also “cancel”


Evaluation and Reasonableness Checks Overview

Complete?

Level of Detail?

Reasonable?

Methodology?

Source?

Sensitive?

Documentation of calibration?

Valid for base year?

Current?

Reasonable?


  • CALIBRATION and VALIDATION are sometimes confused.

  • Model development is sometimes called calibration or estimation as we are estimating parameters and constants for the particular model structure.

  • estimating is a statistical process … want high correlation coefficients and significant parameter values

  • can "import" a model - or borrow structure and parameters from a "similar" area

  • VALIDATION is checking if the model is accurately estimating traffic volumes by calculated measures (like RMSE)

Feedback Loop


Model validation
Model Validation

  • Validation of new model

    • Model applied to complete model chain

    • Base year model compared to observed travel

    • Judgment as to model suitability, return to calibration if not

  • Validation of a previously calibrated model

    • Compare to a new base year, with new …

      • SE data

      • Special gen.

      • Network

      • Counts

“Transportation Conformity Guidelines” (Air Quality) require model validated < 10 years ago


Validation suggestions

  • - Systemwide

  • - compare traffic counts across …

  • - Screenlines

    • (long lines, check major flows) check trip interchange (distribution) between large sections or quadrants

    • need a survey; local knowledge of commute patterns helps

  • - Cordon lines (surround a major generator, e.g. university, CBD...)

  • - Cutlines (shorter, verify corridor flows, fine tuning)

  • if "importing" should validate all borrowed parameters and constants


IT IS VERY IMPORTANT TO HAVE A GOOD COUNT PROGRAM DESIGNED TO SUPPORT VALIDATION!

iowadotmaps.com


To "calibrate" the model, need an OD database from a survey. This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)

The Calibration and Adjustment manual is not intended to replace good OD data, and is intended more for small urban areas. (and has some old data in it! – more recent data area available in the Barton-Ashman publication).


Calibration and Adjustment Steps: This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)

1) verify network and socioeconomic data

2) run the model

3) develop region-wide values (e.g. trips/person, vmt/person)

4) compare region wide values with “Appendix A” values

5) develop screenlines and cutlines

6) compare model results with ground counts for crossings

7) determine problems (system level, local, combination)

8) modify one or more equations, parameters or variables according to chapters on:

- networks

- trip generation

- auto occupancy

- trip distribution

- traffic assignment


Other chapters focus on: This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)

- transit

- external stations

- system vs. local changes

- expected vs. required accuracy

- conclusions

- trouble shooting


Network data reasonableness checks
Network Data Reasonableness Checks This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)

  • Check Trees for 2-3 major attractions*

  • Check coded facility types – how used (BPR?)?

  • Verify speed and capacity look-up table (what LOS used for capacity?)*

    • Speed adjust (can lower the freeway speed if it is being overloaded – tweak?)

  • Significant transportation projects – narrative included? Still viable?

  • Consistency with MTP

  • Plot (facility types, # lanes,

    speeds, area types) to detect

    coding errors*

    * Items we can check in labs


Details This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)

2. Network Errors

2.1 Centroid Connectors

- represent local streets

- check access (all 4 sides?)

- not connected to intersections

- make sure they are not blocked by a physical barrier (river, etc.)


Des Moines Model Capacity Look-up Table This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)


Des Moines Model Capacity Look-up Table (cont.) This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)


Des Moines Model Capacity Look-up Table (cont.) This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)


Des Moines Model Capacity Look-up Table (cont.) This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)


  • 2.3 Intersection Penalties (check them!) This is time consuming and expensive. Few, if any cities have developed OD databases since 1980, but many have updated old ones since then using a small survey (e.g. 1%)

  • - most congestion here

  • - more important in sub-area modeling

  • - turn penalties

  • account for congestion

  • - speed volume function

  • - can include delay on approach links

  • - can do it manually for small networks

  • check for circuity (correct with small turn penalties!)



  • 2.4 Intrazonal times Volume Dependent Turning Delays”

  • increasing intrazonal trips (in distribution) decreases interzonal trips (useful if too many trips are being loaded on the network)

  • number of trips is a function of travel time (gravity model)

    • can adjust travel time on intrazonals

    • can adjust friction factor curve to produce more shorter trips (which intrazonals usually are)

    • can change definition of zones (size, land use)

  • Air quality analysis implications???


3.1 Trip generation Volume Dependent Turning Delays”

- socioeconomic data can be a source of error

- initial step is to check system trip totals, compare w/ Table 4 and A1 and A2 (next pages)

- if there is a problem, check the system number of dwelling units

- still a problem?, check production/attraction rates


Trip generation calibration
Trip Generation Calibration Volume Dependent Turning Delays”

Reasonableness checks – compare to other cities, check future trends

  • Population 503,345

  • Households 201,116

  • Average Household Size 2.50

  • Basic employment 76,795 (33%)

  • Retail employment 50,465 (24%)

  • Service employment 101,697 (43%)

  • Military employment 42,800

  • Population per employee 1.81

  • Person trips per person 4.26

  • Person trips per household 10.65

  • HBW attractions per employee 1.44

  • HBW productions per household 1.74

  • HB shopping attractions per retail employee 5.99

Colorado Springs 1996 Travel Demand Model Calibration


Table A2 Volume Dependent Turning Delays”


More recent data … Volume Dependent Turning Delays”

From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN


3.2 Income Volume Dependent Turning Delays”

- be sure you are using “real” dollars


3 3 p and a rates
3.3 P and A rates Volume Dependent Turning Delays”

  • Problems: old, borrowed, small survey

  • may work OK at the system level, but not for sub-areas

  • check system-wide values (see tables, next pages)

    • raise or lower trip generation rates

    • Person trip or vehicle trip rates used?

      • we usually have person trip by purpose, but can apply occupancy factor and check against vehicle rates (ITE)

  • later, screen line counts can be adjusted by varying trip generation rates (post assignment)

  • check cutlines and cordon counts

  • coordinate all of the above


Non- Volume Dependent Turning Delays”

home


Trip generation calibration typical values
Trip Generation Calibration Volume Dependent Turning Delays”Typical Values

More recent data …

  • Person trips per household: 8.5 to 10.5

  • HBW person trips per household: 1.7 to 2.3

  • HBO person trips per household: 3.5 to 4.8

  • NHB person trips per household: 1.7 to 2.9

  • HBW trips: 18% to 27% of all trips

  • HBO trips: 47% to 54% of all trips

  • NHB trips: 22% to 31% of all trips


Trip generation reasonableness checks
Trip Generation Reasonableness Checks Volume Dependent Turning Delays”

  • Examine trip production and attraction models

    • Form?

    • sensitivity?

    • IMPORTANT: keep parameters reasonable (e.g. don't use negative coefficients in regression models just because they provide the best fit.)

      • If you think you need to use unintuitive parameters, check the whole process...

  • Check models for …

    • External-through and external-local trips

    • Truck trips

  • To calibrate trip generation and trip distribution, sometimes we may use ...

    • default values from past surveys

    • very limited new surveys

    • census journey to work data (CTPP)


Examine trip purposes used use more trip purposes
Examine trip purposes used … Use more trip purposes? Volume Dependent Turning Delays”

TRIP PURPOSES Scaling Factor

HBW low income 0.795

HBW low-middle income 0.823

HBW middle income 0.861

HBW upper middle income 0.908

HBW high income 0.936

HB elementary school 0.733

HB high school 1.991

HB university 0.895

HB shopping 0.698

HB social-recreation 0.945

HB other 0.875

NHB work-related 0.858

NHB other 0.820

Truck 0.985

Internal-external 0.591

Note: each income class is a purpose!

Scale survey for participation (relative participation)

Colorado Springs 1996 Travel Demand Model Calibration


Travel survey data reasonableness checks
Travel Survey Data Volume Dependent Turning Delays”Reasonableness Checks

  • Determine source of travel survey data

    • Types of survey conducted

    • Year of survey

  • Scale survey for participation

  • If no survey (borrowed)

    • Check source of trip rates, lengths, TLFD

    • Is area similar

      • Geographic area?

      • pop/HH/empl. characteristics?

      • Urban density and trans system?

  • Compare to similar regions and to same

    region in earlier times:

    • Person trip rates by trip purpose

    • Mean trip lengths by trip purpose

      • HBW longest? HBO shortest?

    • TLFDs by trip purpose


Socioeconomic data check reasonableness
Socioeconomic Data: Check Reasonableness Volume Dependent Turning Delays”

  • Review source for estimates and forecasts

  • Visualize (plot) trends …

    • Population and household size

    • Household income

    • automotive availability

    • distribution of employment by type (basic, retail, service)

    • employees per household and per capita … rate of increase is decreasing

  • Check future household and employment changes by zone


  • 3.4 Special generators Volume Dependent Turning Delays”

  • e.g. universities, airports, malls, ...

  • Use ITE or survey


3 5 trip balancing factors 4 0 auto occupancy
3.5 trip balancing factors Volume Dependent Turning Delays”4.0 Auto occupancy

  • initially, Ps and As should balance to should be 0.9 to 1.1; if not, check your PA rates and socioeconomic data

  • NHB is usually out of balance

  • Automobile occupancy

    • by trip purpose?

    • Basis?

    • Constant?

  • see table 6 and A9 (next pages … are these still good?)


  • 5.0 Trip Distribution Volume Dependent Turning Delays”

  • 5.1 Mean Trip Length

  • - recall: shape of curve affects trip length distribution

  • See below for effect of changing friction factors



  • 5.2 Estimate Trip Length Volume Dependent Turning Delays”

  • compare average trip lengths (in minutes) by purpose to:

  • HBW t = 0.98 x p.19

  • HBSR t = 2.18 x p.12

  • HBSh t = 8.1

  • NHB t = 0.63 x p.20

  • where p is population

  • SR = social/recreation

  • Sh = shopping


From Volume Dependent Turning Delays”Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN


Source: Volume Dependent Turning Delays”Virginia Travel Demand Modeling Policies and Procedures Manual


Source: Volume Dependent Turning Delays”Virginia Travel Demand Modeling Policies and Procedures Manual


Source: Volume Dependent Turning Delays”Virginia Travel Demand Modeling Policies and Procedures Manual


5.3 Employment Distribution Volume Dependent Turning Delays” Problems (large cities, mostly)

problem: match low income households with low income jobs

solution #1: disaggregate trip purposes by income quartile

solution #2: use k-factors (trial and error) … yuk

Jobs/Housing

Imbalance!



Trip distribution reasonableness checks
Trip Distribution Reasonableness Checks Volume Dependent Turning Delays”

Examine …

  • Mean trip length (increasing or decreasing?)

  • TLFDs

  • Treatment of friction factors (same?)

  • Treatment of terminal times (logic?)

  • Treatment of K factors

  • Comparison with JTW trip length

  • Comparison with JTW sector interchange volumes or percentages.


Calibrating friction factors
Calibrating Friction Factors Volume Dependent Turning Delays”

1st iteration

Calibrate friction factors


Calibrating a Gravity Model Volume Dependent Turning Delays”

Adjusting Friction Factors


2 Volume Dependent Turning Delays”nd iteration


Trip distribution calibration and validation
Trip Distribution Calibration and Validation Volume Dependent Turning Delays”

  • Check modeled vs. household survey TLFD and mean trip lengths

  • Get HBW area-to-area flows from JTW

    HBW 1990 JTW TLFD and Area-to-Area Flows for Kansas City


OD validation Volume Dependent Turning Delays”

Using cell phone and/or GPS location to determine travel patterns is nothing new. But leave it to Google to make it really easy - maybe too easy.

http://googleblog.blogspot.com/2009/08/bright-side-of-sitting-in-traffic.html

Adam Shell

Office of Systems Planning

Iowa Department of Transportation

Link

POA: price of anarchy (30%?)

Nash equilibrium vs. system optimality

OD data are destroyed! (privacy)


6.0 Traffic assignment Volume Dependent Turning Delays”

6.1 All or nothing

- adjusting link speeds will change assigned volumes

- initial speeds should be set to LOS C speeds (0.87 x free flow speeds)


6.2 capacity restraint Volume Dependent Turning Delays”

- volume = f(time)

- final volume is average of all iterations or later iterations can be weighted more heavily

- adjust free flow time or c (capacity) to change volumes

IF… THEN…

Link Speed Travel Assigned

Capacity Time Volume


6.2.1 definition of capacity Volume Dependent Turning Delays”

design: LOS C (0.87c)

ultimate: LOS E (1.00c)

parameters differ depending on definition of capacity …

if defined as LOS C, 0.15(v/c)4

if defined as LOS E, 0.80(v/c)4 (see HCM)


6.3 equilibrium Volume Dependent Turning Delays” - multiple paths may be selected

6.3.2 free speeds in systems with good progression should be coded at about 1.1 times the speed limit time

- more than 10 iterations may be needed for small areas


7.0 Transit Ridership Volume Dependent Turning Delays”

- for small/medium cities, may not have to build a transit network

- If not using a transit network, can use the following method (if trip generation includes transit trips):

1. increase auto occupancy by transit percentage (e.g. if auto occupancy is 1.05, then change to 1.05 x 1.38 = 1.45) if transit percentage is 38%

2. decrease trip production or attraction rates (one of them only, then balance) … if you use productions, can vary mode split by income class

3. modify productions or attractions by zone

- get data from transit company

- adjust socioeconomic data or make direct P/A adjustments


Mode split reasonableness checks
Mode Split Reasonableness Checks Volume Dependent Turning Delays”

  • Mode split model?

    • Form?

    • Variables included in the utility functions?

      • Coefficients logical?

      • Value of time assumptions

      • Parking cost assumptions

  • How do mode shares change over time?

  • Mode share comparisons

    with other cities

http://www.bts.gov/publications/journal_of_transportation_and_statistics/volume_08_number_02/html/paper_05/figure_05_03.html


Mode split calibration and validation
Mode Split Calibration and Validation Volume Dependent Turning Delays”

  • Experienced planning consultant required …

    • Form of LOGIT model

    • Variables included in utility functions

    • Calibration of coefficients for utility function variables

    • Testing for IIA properties

    • Analysis of household survey data

    • Analysis of on-board transit survey data

  • Calibration tasks we can do:

    • Compare highway and transit trips

      • Total

      • By purpose

    • Compare Ridership by route

    • CBD cordon line survey (if bus service is downtown only)


  • 8. External stations Volume Dependent Turning Delays”

  • - externals have no socioeconomic data

  • - Ps and As are prepared by matching ground counts

  • - I/E treated with the gravity model

  • E/E

  • - compare with Table 11 below


  • System vs. local checks Volume Dependent Turning Delays”

  • check 1. system wide (screenlines)

  • 2. major movements (cutlines)

  • 3. links

  • if all screenlines are high or low, vary

  • - auto occupancy

  • - trip generation rates

  • - trip lengths

  • - intrazonal times - all zones

  • - socioeconomic data - all zones

if corridor volumes are high or low, vary (for zones affecting corridor…)

- auto occupancy

- trip generation rates

- intrazonal travel times

- land use

- centroid connectors

- intersection penalties

if links are high/low, vary

- speed

- intersection penalty

- centroid locations

- special generators

- local network configuration


  • 10. Expected/Required accuracy Volume Dependent Turning Delays”

  • We are concerned about errors that would require a design change (e.g. number of lanes)

  • Note that ground counts also contain error

  • Perfectly calibrated models produce link estimates with 1/3 above the standard error in ground counts and 2/3 below the standard error.

  • Need ground counts for 65% of freeways and arterials, and a good sample from other facilities

From Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN


  • 10. Expected/Required accuracy (cont.) Volume Dependent Turning Delays”

  • The correlation coefficient should be greater than .88

  • VMT estimate (region-wide) should be within 5% (take care to compare same roads in systems)

  • VMT/person should be 17-24 for large areas, 10-16 for smaller areas (see also Table A7, next page)

  • VMT/household should be 40-60 for large areas, 30-40 for smaller areas


From CTRE Employment Data Project: Volume Dependent Turning Delays”


From Volume Dependent Turning Delays”Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN


Source: Volume Dependent Turning Delays”Virginia Travel Demand Modeling Policies and Procedures Manual


From Volume Dependent Turning Delays”Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN


From Volume Dependent Turning Delays”Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN


Source: Volume Dependent Turning Delays”Virginia Travel Demand Modeling Policies and Procedures Manual


From Volume Dependent Turning Delays”Minimum Travel Demand Model Calibration and Validation Guidelines for the State of TN


Trip assignment reasonableness checks
Trip Assignment Reasonableness Checks Volume Dependent Turning Delays”

  • All-or-nothing assignment

    • study effect of increasing capacity

    • Compare to Equilibrium assignment

  • Check volume delay equation (BPR parameters)

  • Compare

    • screen line volumes

    • Cut line volumes

  • Time-of-day assignments?

    • Source of factors

    • Peak spreading used for future?

    • If not, conversion factors source?

      (peak hour to 24-hour)

  • Local VMT (% assigned to

    intrazonals and centroid connectors

Equil

ibrium

All or

Nothing


Trip assignment calibration and validation
Trip Assignment Volume Dependent Turning Delays”Calibration and Validation

Assignment calibration performed

last

  • Overall VMT or VHT check

  • 40 to 60 miles per day per HH in large metro areas

  • 30 to 40 miles per day per HH in medium metro

  • +/- 10% OK on screen lines

  • Sign is important


  • Compute by … Volume Dependent Turning Delays”

  • volume group

  • facility type

  • transit assignments

  • time of day


12. TROUBLE SHOOTING Volume Dependent Turning Delays”


Other factors impacting forecasted travel demand
Other Factors Impacting Forecasted Travel Demand Volume Dependent Turning Delays”

  • Can be implied in travel surveys (but not explicit)

    • Telecommuting

    • Flexible work hours

    • HB business

  • How to account for …

    • Aging population

    • Internet shopping

    • Roadway congestion (will it affect generation in the future)

    • New modes


Issues for modeling
Issues for modeling Volume Dependent Turning Delays”

  • Transferability of parameters

    • More research is needed

  • Forensic analysis

    • How well did the models work?

  • Confidence and Credibility

    • How to improve

  • “Official” versions vs. what-if models

    • Integrity of the model

  • Need more transparency, documentation, appropriateness of techniques


ad