NEMSIM: Practical Challenges for Agent-based Simulation of Energy Markets

1. NEMSIM: Practical Challenges for Agent-based Simulation of Energy Markets George Grozev and David Batten CSIRO Manufacturing and Infrastructure Technology Complex Systems Science and CSIRO: Into the Future Rydges Hotel, Melbourne 10-12 August 2005

2. Presentation Overview • History and concepts (NEM as a CAS) • NEMSIM overview and key features • Practical challenges for agent-based simulation

3. Brief Historical Review of the Project • July 2002: A postdoc position awarded. Dr. Xinmin Hu started in Nov. 2002. • January 2003: Commenced as a CSS project: “Top-up” funding from CSIRO’s Centre for Complex Systems Science • October 2003: Commenced as a Theme 1 project in CSIRO’s Energy Transformed Flagship Program • April 2004: Flagship Science e-Seminar Series 2: Energy Transformed (John Wright, David Batten) • April 2005: NEMSIM Industry Focus Group Meeting (Mercure Hotel, Melbourne)

4. NEMSIM: National Electricity Market Simulator • Agents in NEMSIM: • 27 Scheduled Generator Companies • 12 Non-scheduled Generator Companies • 20 Network Service Providers • 29 Market Customers • 9 Traders • An Independent System Operator (NEMMCO) • Potential Clients: • Regulators (ACCC, AER, AEMC) • Government (DITR, SA, Tasmania) • TNSPs (Powerlink, Transgrid) • Customers (EUAA, ERAA, Origin, AGL)

5. The NEM is a Complex Adaptive System • Evolving markets on an interconnected grid • about 100 interacting, autonomous agents (firms) + others • about 300 grid-connected, generating units. • Agents are intelligent, adaptive & behave differently • pursue goals unique to their firms’ interests • make decisions on the basis of their own knowledge/beliefs • change strategies in the light of their and others’ experiences. • No agent knows what all the other agents are doing • each agent has access to only a limited amount of information. • Some act more conservatively than others • e.g. they are more constrained (e.g. by debt) than others.

6. Our Science – Agent-based Simulation • Equations-based models • too static, aggregate or stylized to handle this complexity • Agent-based simulation • computational experiments • software agents, environments and rules • agents learn and adapt strategies over simulated time • evolutionary computation and equations-based methods • can explore impacts of rule changes before their introduction • can evolve adaptive responses of competitors • collective outcomes can be unexpected, even undesirable

7. Presentation Overview • History and concepts (NEM as a CAS) • NEMSIM overview and key features • Practical challenges for agent-based simulation

8. NEMSIM Overview

9. NEMSIM Overview - continued

10. NEMSIM – Generating Units Displays Bid Stacks Dispatch Revenue GHG Emissions

11. Key Features • Includes all key players in the NEM • Models individual agent’s behaviour • Weather model and data from 100 years • Wholesale market model and extending to other markets, e.g. contract market • Potential effect of distributed generation • Transmission modelling • Bid strategies – e.g. lookaheads • Scenario investigations – new plants, maintenance, emergency shutdown, blackouts, new rules • Scenario comparisons • Reports – dispatch, revenue, CO2, by regions, by companies, by plants, weekly, monthly, yearly • Environmental markets, e.g. carbon trading

12. Other Important Features • XML editor • Simulation time control • Lookaheads • Scenario comparison • Distributed generation • New plants • Maintenance & shutdown • Reports

13. Area of Applications • Short-term trading • analyse market bidding data • analyse “what-if” bidding scenarios • Medium-term hedging and contract markets (retailers, generators) • Long-term investment (new generators, transmission lines, distributed generation, renewables) • Greenhouse gas emissions estimates • Carbon trading (when rules are proposed) • Explore the impact of new technologies, new market rules, new grid structures, new participants

14. Presentation Overview • History and concepts (NEM as a CAS) • NEMSIM overview and key features • Practical challenges for agent-based simulation

15. Selection of Agent-based Simulation Platform • Develop our own platform • EMCAS - Argonne National Lab • DIAS/FACET – Argonne National Lab • RePast • Swinburne’s simulation framework – agent implementation of the Victorian Gas Market

16. Other Practical Challenges for NEMSIM • Adequately reflecting all the subtleties inherent in market-to-network interdependencies (DITR) • Developing efficient heuristic algorithms for interactive decision-making • e.g. adaptive learning procedures • e.g. multi-criteria decision-making • Distinguishing between counterintuitive results and programming errors • Keeping running times reasonable while adding more dynamic features • Developing confidence and trust among potential users and the market operator (NEMMCO)

17. Challenges of Learning in the NEM • Depending on their own competitive position, each generator behaves differently • Bidding strategies differ between states, but even more so between generators within states • Although strategies differ, we may be able to develop a generic bid function for all of them (just varying parameters/markups) • Most generators change bid capacities, occasionally changing bid prices (or price increments) • Thus each firm that owns generating units will need to be examined, if we wish to approximate reality

18. Potential Learning Algorithms • Genetic algorithms (see e.g. Goldberg, 1989, Mitchell, 1998, Chattoe, 1998, Dawid, 1999) • Genetic programming (see e.g. Koza, 1992) • Reinforcement learning algorithms (see e.g. Erev and Roth, 1998; Sutton and Barto, 1998) • Q-learning (see e.g. Watkins, 1989; Tesauro and Kephart, 2002) • Classifier systems (see. e.g. Holland, 1992) • Learning algorithms for automated markets (see e.g. Gjerstad and Dickhaut, 1998; Tesauro and Kephart, 1998)

19. LOOK SIDEWAYS (Bidding rules) NEMSIM agents can look ahead, sideways and back LOOK AHEAD (Strategy evaluation) • Own unit availability • Price trends/peak loads • Hedging strategy • Weather • Load forecasts • Competing unit availability • Competing bids • Market rules TIME Agent • Bid acceptances/rejections • Unit utilization • Unit profitability • Market price vs. bid price • Weather and Load LOOK BACK (Short and Long Term Memory)

20. Look-aheads in NEMSIM • Agents have look-ahead capabilities • Run the simulation forward for various periods • Test & compare a range of available strategies and plans • Agent adopts strategy showing best possible outcome • Strategies retested at start of each new period • Plans/strategies changed to counter changes of others • Does a look-ahead capability add value to the existing (comparative static) approaches?

21. Value of a Look-ahead Capability

22. FG Meeting: Challenges for NEMSIM • Focus more, refine agents adaptive behaviour • How agents think and interact, not just bid • Explore demand-side management options • Locational issues • Customers as agents • Differentiate between short and long-term • Treat GHG/carbon tax/emissions trading • Explore DG/wind/green power • Talk to appropriate potential users • Regulators • Government policy makers • Network companies

23. Practical Advantages of NEMSIM • Practical application of a CSS methodology • To a real world complex adaptive system (the NEM) • Socio-economic/physical/environmental interactions • Each and every agent’s adaptive behaviour can be represented and modified • Different collective outcomes can be generated and performances compared in advance (“look-aheads”) • Conditions when unattractive outcomes occur (like price volatility & market power) can be identified • This kind of simulation goes beyond the classical simulation models in energy economics • User-friendly human-machine interface

24. Acknowledgments • Research and Development Group • Energy Transformed Flagship • Swinburne University of Technology • CMIT: • George Grozev • David Batten • John Mo • Miles Anderson • Geoff Lewis • Mario Sammut • CMAR: • Jack Katzfey • Marcus Thatcher • UNSW: • Xinmin Hu • Paul Graham – Theme Leader “Energy Futures” • Terry Jones - Theme Leader “Low Emission Distributed Energy” • Prof. Myles Harding • Neale Taylor

25. Thank you