Incentives, Regulation and Analysis for Efficient ISO Markets

1. Incentives, Regulation and Analysis for Efficient ISO Markets Richard P O’Neill Chief Economic Advisor FERC the Isaac Newton Institute Cambridge, UK March, 2019 the views expressed are not necessarily those of the Commission, the Chairman, or any individual Commissioner

2. Why study Power Markets • World Consumption (2017): 22,000 TWh • At $100/MWh: cost ≈$2 trillion/year • 10% savings is about $200 billion/year • money can't buy you love

3. outline • History, laws, regulation and evolution • Efficient Energy markets • Capacity markets • Optimal Transmission expansion • Optimal topology • Physics and Computing (years to minutes) • AC Load flow and Generators • Approximations and challenges • No

4. U. S. Legal Historythe 1930s Laws • Pre-1935 electricity was regulated • by municipalities then as they grew, by states • Using cost-based and benchmarked rates • As markets grew, regulatory gap for interstate transactions • 1935 the Federal Power Act and 1938 the Natural Gas Act • Rates (prices) must be just and reasonable and • with no undue discrimination (eg, Ramsey-Boiteux pricing is due) • What arejust and reasonable prices? • Dictionary: conforming to a standard of correctness or • what is morally right or good

5. History of Just (and Reasonable) Prices • Roman law was flexible. • price is "just" if it was agreed to by the contracting parties • 284 A.D. emperor Diocletian set the maximum prices • for beef, grain, eggs, clothing, … • under the penalty of death • 13-15th Cent. Just prices in the Roman church • Economic theology: cost-based v. market-based • Market-based: Thomas Aquinas • Dominicans, Jesuits • Salamanca school • Cost-based: no more than the cost • Franciscans, Francis of Assisi, Biel, Scotus Thomas Aquinas Francis of Assisi

6. The First Century of USjust and reasonable prices • reliability issues, cost-of-service and cost pass-throughs. Averchand Johnson effect on steroids for vertical integrated utilities !!! • Little incentive to optimize the dispatch. • better software for optimal dispatch displaces more capital investment -- arguably a negative incentive • century old neglect stunted the investment in optimization • The result was billions $/yr in forgone cost savings • Benefit/cost of software is usually > 100 (maybe > 1000) • Benefit/cost of hardware is usually < 1.5

7. Power PoolsISO precursors • 1927 utilities in Pennsylvania, New Jersey and Maryland agree to share generating resources and create PJM • 1965 Northeast Blackout, New York’s utilities create the New York Power Pool (NYPP) • to coordinate the reliable operation of their systems • dispatched generation to balance supply and demand, • 1971 New England Power Pool (NEPOOL) established for • central dispatch to improve reliability and economics • joint planning and coordinate of G and T outages • Split-the-savings prices

8. Laws and Orders Post Oil Price Shocks • 1970s oil price shocks • 1978 President Carter’s energy legislation • NPGA partial deregulation of natural gas • Integrates interstate and intrastate markets • Mostly repealed • Public Utilities Regulatory Policy Act (PURPA) • Force purchase of renewables and cogeneration • new entrants/Independent Power producers (IPPs) • Fuel Use Act outlaws the use of natural gas in generation • repealed • Sets the stage for deregulation and open access

9. Economic Literature • 1977 Turvey, Electricity Economics • 1980s Chicago school ascendancy • Coase, ‘marginal cost pricing’ does not work • Demsetz, auctions can act as regulation • Government failure and regulatory capture • 1980s Cambridge (west) school • 1985 Joskow and Schmalensee, Markets for Power • 1988 Schweppe et al, Spot Pricing of Electricity • 1990s Hogan, FTRs, CfDs, Reactive power, scarcity pricing • 2000 Turvey formal marginal cost definition is unrealistic • Defines marginal costs as incremental costs

10. Laws and Orders the advent of markets • 1980s market-based rates are just and reasonable if you have no market power • 1990s Open access as a condition of an electric merger • 1992 FERC orders (No. 636) natural gas open access • 1996 FERC orders (No. 888) electric open access • Point-to-point ‘contract path’ transmission service from vertical integrated utilities or • Form ISO under stated principles • ISO membership as a condition of a merger • markets and competition evolve

11. EIM

13. New TechnologiesNew Issues (2000s) • Solar and wind • Batteries • Electric cars • net load = load-renewables • need • flexible generators • Greater range [min, max] • Fast ramp rate • Price-responsive demand • Smart controllable devices

14. Old Stochastics • Binary generator failure • Demand = f(temperature) • Peak is a hot afternoon in August • Could see it coming. • Off peak ‘peaks’, e.g., polar vortex • For dominant hydro systems (Brazil 75%), • Need energy for N-year drought • Need opportunity cost pricing • Forward energy (not capacity) markets

15. Weather Stochastics • Bad/unexpected weather forecast is the largest contingency • Weather contingencies • 70% of generator failures due in part to weather • Transmission capability due in part to weather • new stochastics • generator = f(temperature, operation, maintenance) • Demand = f(temperature, humidity) • Solar = f(sunshine) • wind = f(wind, shut down at -20oF or max wind) • Hydro = f(rain, snow) • Where and when is the peak? • Cloudy and windless day • Sunny and windy day

16. NewStochastics • For dominant renewables systems • Need energy not capacity • opportunity cost offers • Forward energy markets • Flexibility • Fast ramp and output range • Price-responsive demand • Hydro • Batteries • Better pricing with price-responsive demand

17. Europe History • Government owned • Government set rates • cost-of-service ? • Retail price discrimination Market mechanisms • TSO balancing • Power exchanges • Generation under competition rules • Nash equilibria United States History • Private/muni owned • Just and reasonable rates • cost-of-service • Retail price discrimination Market mechanisms • ISO real-time market • ISO day-ahead market • Generators under just and reasonable prices • Economically efficient markets

18. Europe Externalities • Generous feed-in tariffs • EU carbon prices • Country renewable mandates • Local social cost of retirement (unemployment, tax revenues, decommission) • Local social cost of entry (environment, property rights) United States Externalities • PURPA (1978) feed-in tariffs • State level carbon prices • Federal subsidies for wind and solar • State renewable mandates • Local social cost of retirement (unemployment, tax revenues, decommission) • Local social cost of entry (environment, property rights)

19. ISO Market Design Goals • Primary goal: Maximize market efficiency • includes reliability [Sean: you don’t need prices] • internalize the externalities • auctions are the best way to achieve the goals • Pricing goals: efficient price signals to support sustainable market efficiency. • Secondary goals • Price risk management through bilateral contracting • good market information and liquidity • minimize government intervention • minimize transactions costs • ease of entry and exit

20. ISO Auction Design Properties for Just and Reasonable Rates • Each point in time and space represents a different product • Rates consistent with the physics. • non-confiscation of bids and offers • Revenue adequate • mitigate market power: offer incremental costs • Transparent prices (uplift is not transparent) • incent behavior through pricing that leads to sustainable efficiency

21. Why ISO energy auction markets? • To achieve the efficient dispatch within reliability rules over a large region • Physical bilateral trading takes too long, is expensive and often infeasible • Need balancing mechanism • ISO Auctions • Lower transactions costs • Lower risk (almost zero) of non-delivery • Lower reserves needed • Provide greater transparency • Should signal future decisions • Be compatible with bilateral markets

22. ISO Markets and Planning • Four main ISO Auctions • Real-time: for efficient dispatch (every 5 minutes) • Day-ahead: for efficient unit scheduling (daily) • Generation Capacity: to ensure generation adequacy and cover efficient cost recovery (annual) • Transmission rights (FTRs): to hedge transmission congestion costs (monthly, annual) • Interconnection, Transmission Planning and Investment • Competition, cooperation and eminent domain • All use approximations due to software limitations

23. Monitoring and Mitigation • The network including thermal and voltage constraints creates pockets with market power that change over the day and locations • Market efficiency is achieved with incremental costs offers • Any bid or offer that lowers market efficiency should be considered an exercise of market power • Self-scheduling may be • attempt to exercise market power or • contractual obligation or • due to market design, for example, short time horizon or weak offer structure • Fail-safe Mitigation: mitigate offers to approximate incremental costs

24. ISOs Market Designs Failures • Zonal markets (PJM, ISONE, CAISO) • Einstein: Stupidity is doing same thing and expecting different results • Historic congestion not representative of an optimized system • day-ahead markets (CalPX) • larger but infeasible markets will be more competitive!!! • what is the clearing price for infeasible markets? • Separate reserves markets (CAISO price inversion) • Reactive power is not imaginary and is not free • Free primary frequency response • Prohibit long-term contracts (Cal) • Missing money, market signals, LMP settlements and uplift

25. The Energy Policy Act 2005(551 pages) • promote reliable and economically efficient transmission and generation of electricity’ • study economic dispatch and the potential benefits • More penalty authority to prevent market manipulation • Transmission • rate incentives to promote investment • State transmission siting (4th Circuit decision) • Essentially ratified ISOs (section 210M)

26. Day-Ahead Market • must be solved in three hours or less. • Efficient Unit commitment and dispatch • over 24 financially-settled hourly periods • additional periods to avoid end of horizon effects. • settled financially and physically feasible • Procedure • Offers are mitigated • units are efficiently committed and dispatched (MILP) • AC feasibility is checked if pass stop • if fail add linear constraints and iterate

27. Real-Time Market • Five minute periods • may have a look ahead of up to four hours.  • Only the first period is settled financially. • Procedure • Offers are mitigated • limited number of units are efficiently committed • AC feasibility/reliability if pass stop • if fail add linear constraints iterate. • Redispatch must be solved in five minutes or less • Operator intervention 

28. Reactive Power Marketsis it too cheap to meter? • Reactive power why care it is imaginary !!! • Sources: generators, transmission, and load • For generators, real and reactive power are joint products • Not a public good; it’s a club good • Needed most in big cities • What is the cost of reactive power? • Marginal costs may be low; incremental costs are high • Cost of a cut set or closed loop interface • Startup and minimum operating costs of generators • Suppression of LMP

29. Consumers the missing market participants Can’t boil water with a nega-watt • Most consumers receive very weak price signals • monthly meter; ‘see’ monthly average price • On a peak hour • cost > $1000/MWh • price < $100/MWh • results in market inefficiencies and • inefficient purchase decisions. • Smart meters, dumb prices • Time of use pricing becomes obsolete!!! • Answer: Price-responsive demand • Result: Large two-sided market!!!!!!!!! He's as blind as he can be just sees what he wants to see Alexia make ice when the price is negative

30. Desirable Attributes for a Pricing Mechanism (Chao, PJM, 2018) • Economic Efficiency • Support efficient system operations (optimal dispatch) • Prices reflect scarcities and costs (price-responsive demand) • Incentive compatibility • Promote truthful bids (mitigation and non-confiscation) • Reward efficient behavior (infra-marginal generators make profits) • Revenue sufficiency • Sustain competitive markets (profits cover investments) • Minimize the use of uplift payments (make-whole payment = 0)

31. Convex Pricing Theory • Oversimplified for pedagogy (see Turvey 2000) • In convex markets, LMP is the theoretical pricing goal for market efficiency • marginal cost pricing (LMP) clears a convex market • Entry displaces the output of the marginal supplier • Need two sides

32. Vertical Demand curve No curtailment No scarcity rent for $30 generator $30/MWh LMP = $30 scarcity rents bid $/MWh $20/MWh demand $10/MWh Quantity in MWh

33. Demand exceeds Supply What should generators be paid? between $30 and $8000 (VOLL) $30/MWh bid $/MWh $20/MWh demand curtailment $10/MWh Quantity in MWh

34. Load Curtailment $30/MWh LMP = $30 bid $/MWh $20/MWh demand curtailment $10/MWh Quantity in MWh

35. Price-Responsive Demand • Price-responsive demand in day-ahead and real-time market • Supply ancillary services • Entry when ∆Consumer Surplus > Incremental Cost • LMP is the ‘convex’ margin • No capacity charges • Price Signals are ex-post not a signal to change in the current market dispatch • price-responsive demand and reserves pricing reduces the missing money and need for capacity markets

36. Load Curtailment $30/MWh LMP = $30 bid $/MWh $20/MWh demand curtailment $10/MWh Quantity in MWh

37. The ‘Organic’ Solution Price-responsive demand Price- responsive demand Market clearing price: $35 scarcity rents = (35-20)80 = $1200 sr=$300 scarcity rents (sr) = (35-10)100 = $2500 $30/MWh bid $/MWh $20/MWh $10/MWh 100 180 240 Quantity in MWh

38. Non-Convex Economics • ISO markets are non-convex • AC flow equations are nonlinear • Binary decisions, e.g., start-up and minimum run time • Many approximations, difficult to solve • Efficiency objectives are the same as convex theory • Maximize economic efficiency • Prices for incentives to sustain efficient entry • Convex markets are a special case of non-convex markets • Efficient dispatch and pricing is more complex • Entry and exit conditions are more complex

39. Convex Pricing in Non-convex Markets • Many analogies to convex markets breakdown • Marginal cost pricing (LMP) is not efficient and may • not clear a non-convex market • be confiscatory (revenues do not cover costs) • create losers out of winners • send weak entry and investment price signals • need a make-whole payment • Units at min. operating level have marginal costs ≥ LMP • No ISO uses LMP pricing

40. Pricing Reality • No ISO uses LMP pricing; LMP does not clear the market • Currently all ISO have a two-part pricing mechanism • CAISO and SPP: LMP + make-whole payment • NYISO and PJM: minimum operating level relaxation with make-whole payments • MISO and ISONE: binary variable relaxation with make-whole payments • Make-whole payments may create zero-profit for infra-incremental generators • make-whole payment • is not transparent • ex-post weak allocation of these costs • Some infra-incremental generators are not profitable • What are the economic properties of these methods?

41. PJM energy marketsChao 2018 • Uplift payments are persistent: $200 million a year • 20% of the time, when demand increases energy price falls • Over 60% of generation is self-scheduled • Many CTs submit inflexible bid parameters • Revenues move from the energy to the capacity market • Slowing demand growth, flattening supply curves and penetration of renewables present new challenges

42. non-convex generator • A non-convex generator (fossil or nuclear) has binary • Binary startup costs and minimum operating level (> 0) • min run time and min down time • Failure probability: {0, 1} • Nonconvex cost function • Combined Cycle Combustion Turbine Generator • for each CT, startup costs, minimum operating level, min run time and min down time

43. Combined Cycle Generator Physics Residual demand curve Market surplus

44. Non-convex Pricing in Non-convex Markets • Objective: allocate costs based on causality • Minimize make-whole payments • Sustainable prices • Pricing run • Relax the minimum operating level • include avoidable average incremental costs (at optimal dispatch) in marginal cost function • Allocate costs based on cost causality • Ramsey-Boiteux pricing for LIP > marginal value > mc

45. One-Period Non-convex Auction Market(changes from the convex market in red) λ is the LMP and δiis the make-whole payment

46. One generator AIC curve (eg, CT) Short-run economies of scale b1 AIC b2 Efficient market and LIP for a non-convex market. cAI1 LIP Linear surplus covers fixed costs c1 b3 b4 p* pmin1 pmax1 quantity

47. Multiple AIC curves with pmins at origin b1 AIC c4 b2 Efficient market and LIP for a non-convex market. Gen 3 sets the LIP cAI3 LIP c3 cAI2 cAI1 c2 c1 b3 b4 <-∑pmin-> quantity p* (LIP, pi*) is a non-convex auction equilibrium

48. AIC Pricing Run (linear program)cAIi= ci + csui/pi*highest AIC sets the market price = LIP = maxi {cAIi}

49. Non-convex Dynamic Network Markets • In multi-period markets • If minimum run time constraint binds, allocate costs to the periods that caused the incremental generator to be dispatched • Delete non-binding ramp constraints. pminrelaxation can trigger false ramp rate constraints in the pricing run . • In networks, flowgate marginal price am be a false flag • flowgates with no marginal value but incremental value • Good signal for expansion • Revenue adequate, non-confiscation, incentive compatible

50. What is the role of an ex-post LIP auction prices? • It does not signal a change in dispatch!!!! • LIPs with Ramsay-Boiteux pricing clear the market • Price transparency (NO make-whole payments) • Incentive Compatibility • self-dispatch charged rebalancing costs (liquidated damages) • Reserves price meet the arbitrage condition • infra-incremental units make a profit • Is consistent with a efficient bilateral market • Gives better price signals for • price-responsive demand, • short-term entry and • Investment (may not be submarket perfect)