Dalrp static baseline analysis
This presentation is the property of its rightful owner.
Sponsored Links
1 / 34

DALRP Static Baseline Analysis PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

DALRP Static Baseline Analysis. Miriam Goldberg Presented to the NEPOOL Markets Committee Meeting March 31, 2008. Background. On January 23, 2008, ISO-NE presented several issues with the Day-Ahead Load Response Program to the NEPOOL Markets Committee.

Download Presentation

DALRP Static Baseline Analysis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Dalrp static baseline analysis

DALRP Static Baseline Analysis

Miriam Goldberg

Presented to the NEPOOL Markets Committee Meeting

March 31, 2008



  • On January 23, 2008, ISO-NE presented several issues with the Day-Ahead Load Response Program to the NEPOOL Markets Committee.

  • At that time, ISO-NE proposed indexing the DALRP Minimum Offer Price to the Forward Reserve Fuel Price Index using an implicit heat rate of 12.92 MMBtu/MWh.

  • The NEPOOL Markets Committee voted to implement the change effective February 8, 2008.

  • ISO-NE committed to a 60-day stakeholder process to evaluate alternative proposals.

Background continued

Background (Continued)

  • Three Market Participants submitted alternative proposals to ISO-NE in February 2008.

  • ISO-NE retained KEMA to conduct an independent analysis of the key alternative proposals.

Kema experience and qualification

KEMA Experience and Qualification

  • California Energy Commission: Protocol Development for Demand Response Calculation – Findings and Recommendations

  • ISO-NE: Demand Response Reserves Pilot Technical Support

  • NYSERDA: Technical Support for Demand-Responsive Load Pilot

  • Gruenich Associates: Technical Support for California Demand-Response Program

  • EPRI: Methods Assessment for Load Profiling for Retail Market Settlement

Alternative proposals evaluated

Alternative Proposals Evaluated

  • IECG Proposal – Option 1:

    • Lower the effective heat rate below the present 12.92 MMBtu/MWh level.

    • IECG did not object to EnerNOC alternative proposal described below.

  • EnerNOC Proposal:

    • DALRP Minimum Offer Price restored to $50/MWh.

    • Implement a New Customer Baseline methodology for all ISO Load Response Programs.

      • Under EnerNOC’s proposal, the Customer Baseline is calculated using a minimum of 15 programs days over a trailing 45 program-day window using a weighted-average methodology.

Analysis methodology

Analysis Methodology

Analysis objective

Analysis Objective

  • Determine how accurate each Customer Baseline methodology is at predicting a customer’s actual hourly load with no load reduction event.

  • Does the baseline tend to be high or low, and by how much?

  • How variable is the difference between the estimate and the actual load?

Analysis approach

Analysis Approach

  • Data analyzed

    • Actual 2007 hourly loads from a sample of customers (assets) that DID NOT participate in the DALRP or Real-Time Price Response Program.

  • Hourly Customer Baseline calculations for each asset

    • ISO-NE’s current methodology.

    • Each Alternative Proposal.

    • “Weather adjustment” to the calculated Customer Baseline consistent with ISO-NE’s current methodology.

Measuring baseline error

Measuring Baseline Error

  • In each DALRP cleared hour, for each asset, calculate the difference between the asset’s adjusted Customer Baseline and its actual load.

  • Looking only at NONPARTICIPATING assets.

  • Difference between baseline and actual is baseline error for those assets and hours.

  • Summaries of baseline errors over customers and hours measure overall baseline method accuracy.

Data sources

Data Sources

  • ISO-NE provided hourly data from 533 non-participant commercial and industrial customers (assets).

  • The data used in the analysis included all assets that had a full year of data and no missing hours.

Determining cleared hours for the analysis

Determining Cleared Hours for the Analysis

  • Assets are assumed to bid at minimum offer price every eligible day after an assumed starting date.

    • Evaluated different starting dates.

  • Offer clears when minimum offer price is less than or equal to Day Ahead LMP.

    • Based on actual 2007 Day Ahead LMPs in the Load Zone where each asset is located.

  • Each event started with the first cleared hour after 7am, and ended at 6pm.

Baseline error

Baseline Error

  • Error = (Baseline) – (Actual Load)

  • Positive error means baseline is over-estimating actual load.

  • Therefore potentially overstating load reduction.

Median relative error

Median Relative Error

  • Measures bias or systematic error

  • Error = (Baseline) – (Actual Load)

  • Monthly, asset-level relative error = (mean asset error)/(mean actual Load)

    • Allows comparison across assets of varying sizes

  • Median relative error = Median of monthly relative errors over assets

    • by month or day

Asset specific hourly error error baseline actual load

Asset Specific Hourly ErrorError= (Baseline) – (Actual Load)

Dalrp static baseline analysis

Asset Specific Relative ErrorRelative Error = Average Error/Average LoadMonthly Relative Error uses averages over all cleared hours in a month.

Median monthly relative error

Median Monthly Relative Error

  • Calculated 533 asset-level relative errors for each month.

  • Identified the Median Relative Error in each month.

    • Equal number of assets have relative errors above and below the median.

  • Because error is in percentage terms, all assets are equally weighted.

  • Median is less affected by outliers than Mean.

Starting date matters

Starting Date Matters

  • Market Participants can begin offering an asset into the DALRP on any non-holiday weekday (after baseline is established).

  • KEMA’s analysis shows that the Relative Error of all the Customer Baselines is dependent on the date which an asset starts offering in the DALRP.

  • KEMA evaluated 4 different offer start dates:

    • January 2007

    • April 2007

    • August 2007

    • September 2007

First offer date january 2007

First Offer Date: January 2007

First offer date april 2007

First Offer Date: April 2007

First offer date august 2007

First Offer Date: August 2007

First offer date september 2007

First Offer Date: September 2007

Comparing alternative proposals median monthly relative error dalrp offers start august 1 2007

Comparing Alternative ProposalsMedian Monthly Relative Error, DALRP Offers start August 1, 2007

Relative root mean square error rmse

Relative Root-Mean-Square Error (RMSE)

  • Measures typical error magnitude

  • Combines systematic bias and variability

  • Relative RMSE is always positive

  • Greater relative RMSE indicates greater bias and noise (variability) in the error estimate.

  • RMS = Square root of average squared value

    • square each value, average the squares, take the square root of the average

  • Relative RMSE= RMS Error / RMS Load

Comparing alternative proposals median relative rmse dalrp offers start august 1 2007

Comparing Alternative ProposalsMedian Relative RMSE, DALRP Offers start August 1, 2007

Number of cleared days

Number of Cleared Days

  • Proposals have different Minimum Offer Prices.

  • Minimum Offer Price compared to actual 2007 Day Ahead LMPs.

  • At $50/MWh, all eligible days clear.

  • As Minimum Offer Price increases, fewer eligible days clear.

Number of cleared eligible days continued

Number of Cleared Eligible Days continued

The number of cleared days in this table is based on 2007 Connecticut Day Ahead LMPs.

Monthly total mwh error

Monthly Total MWh Error

  • The sum of all cleared hourly errors (Baseline – Actual Load) for the month across all assets.

  • Negative errors (hours when Actual Load is greater than the Baseline) are included in the sum.

  • Unlike median relative error, assets are not all weighted equally.

Comparing alternative proposals monthly total mwh cbl mw dalrp offers start august 1 2007

Comparing Alternative ProposalsMonthly total MWh Δ (CBL-MW), DALRP Offers start August 1, 2007

Monthly sum of payments

Monthly Sum of Payments

  • All cleared hourly errors are monetized at the hourly Day-Ahead LMP.

  • Hourly payments are summed across all hours in the month and across all assets.

Comparing alternative proposals monthly sum of payments dalrp offers start august 1 2007

Comparing Alternative ProposalsMonthly Sum of Payments (+/-), DALRP Offers start August 1, 2007

Lessons learned

Lessons Learned

  • Various measures of error tell similar stories for this study.

  • Relative RMSE results mirror relative error results.

    • That is, most of the baseline error is systematic, not noise.

  • Monthly total error results are driven by combination of error magnitude and number of cleared days.

    • Same general trends found as for relative measures.

    • Indicates similar patterns across asset sizes.

Conclusions on baseline options

Conclusions on Baseline Options

  • Original DALRP design ($50/MWh Minimum Offer Price) has highest overall error, by all measures of error.

    • Equivalent to very low heat rate or clearing every day.

  • Current DALRP design (Minimum Offer Price indexed to 12.92 MMBtu/MWh) has lowest overall error.

  • Minimum Offer Price indexed to 10.85 MMBtu/MWh (lowest examined) distinctly worse than others.

  • Other heat rate options evaluated fall in between.

Conclusions continued


  • EnerNOC option

    • Up to two months after starting point errors are similar to original DALRP design.

    • By 4th month after starting point errors are similar to current DALRP design.

Thank you for your attention

Thank you for your attention.

  • Login