Analysis of Energy System by Energy Model Formulated as Multi-agent Simulation

1. Analysis of Energy Systemby Energy ModelFormulated as Multi-agent Simulation IEW2005July 5th-7th, 2005 Yumiko WATANABE, Takeshi SHINOHARA, Taketo HAYASHI, Yasumasa FUJII, Kenji YAMAJI the University of Tokyo, Japan

2. Outline of Presentation • Objectives • Methodology -Formulation of models- • Common structureof our global energy models • Formulation of conventional optimization energy model • Formulation of new agent-based energy model • Analysis results • 4 case settings for calculations • Difference of results between 4 cases • Summary

3. Outline of Presentation • Objectives • Methodology -Formulation of models- • Common structureof our global energy models • Formulation of conventional optimization energy model • Formulation of new agent-based energy model • Analysis results • 4 case settings for calculations • Difference of results between 4 cases • Summary

4. Objectives To develop agent-based global energy model And To conduct preliminary analysis • Conventional global energy model • Formulated as optimization model • Suitable to analyze global energy policies • Problem: Unsuitable to analyze conflicts of domestic policies such as energy security and emission trading →New formulation of energy model is needed

5. Outline of Presentation • Objectives • Methodology -Formulation of models- • Common structureof our global energy models • Formulation of conventional optimization energy model • Formulation of new agent-based energy model • Analysis results • Case settings for calculations • Calculation results • Summary

6. Common Structureof Global Energy Models (1) • To calculate energy supply system under given demand based on SRES B2 Scenario • Succeeding DNE21 model (Fujii and Yamaji, 1998) • Geographical coverage:Whole world disaggregated into 54 regions, 82 nodes • Time horizon (original) :2000-2100 at 10-year intervals City nodes (production, conversion, consumption) Production nodes (production, conversion) Geographical Coverage

7. Common Structureof Global Energy Models (2)

8. Formulation of Conventional Energy Model • Formulated as optimization calculation • Objective to minimize global energy system cost • Huge but simple linear programming • Facility costs • Operation costs • Efficiency data • Other factors • Resource reserve • Demands Input Data • Objective function • Constraints: • Resource constraints • Balance constraints Optimization Calculation Output Variables • Production • Conversion • Transportation • Consumption • Facility • CO2 emission

9. Formulation of NewAgent-based Global Energy Model • Agents: • Have objective to maximize own reward • Find strategy through iterated calculation (reinforcement learning) • Take action up to the strategy • Market: • Allocate energy system to agents with optimization calculation AGENT AGENT MARKET Input Data Strategy Act AGENT Update Allocating Calculation Objective Reward Output Variables

10. Formulation of NewAgent-based Global Energy Model • Assumption for preliminary analysis • Target year: 2010 (static decision making) • Focused attention to Oil trading • Agents: Groups of regions (ex. OPEC, EU, …) • Objective: To minimize their energy system cost • Strategy: To manipulate oil trading priceTo charge premium on oil export / tariff on oil import +Premium Agent 2 +Tariff Region A Region E Region G Region B Region D Region F Region C Region H Agent 1

11. Outline of Presentation • Objectives • Methodology -Formulation of models- • Common structureof our global energy models • Formulation of conventional optimization energy model • Formulation of new agent-based energy model • Analysis results • 4 case settings for calculations • Difference of results between 4 cases • Summary

12. Case Settings for Calculations • 4 cases with different sets of agents 1 agent 2 agents 6 agents Regional names were given for convenience (not corresponding to the actual groups of nations) 8 agents

13. Calculation ResultsProgress of Learning Oil Price Strategies • The price strategy in 1 agent case is zero • As strategical act, • OPEC charges premium price on its export • EU and Japan/Korea impose tariff on import • Positive value: • Oil tariff on import • Negative value: • Oil premium on export 1 agent 2 agents 6 agents 8 agents

14. Calculation ResultsTotal Energy System Cost • Total cost in 1 agent case is the lowest(corresponding to conventional model) • With strategic acts, energy system becomes inefficient • With enough agents, energy system might recover efficiency Case setting

15. Calculation ResultsAmounts of Oil Production • Oil production in 1 agent case is the highest • With strategic acts, total oil production decreases • OPEC’s share decreases andLatin America and Africa’s share increase Case setting

16. Calculation ResultsWorld Oil Price • World oil price appears as export shadow price of OPEC • Oil price is 22[$/bbl] in 1 agent case • Oil price shifts due to agent’s strategic action, andbecomes higher by4[$/bbl] in 6 and 8 agents case • Domestic oil prices in importing nations are higher by their tariff

17. Outline of Presentation • Objectives • Methodology -Formulation of models- • Common structureof our global energy models • Formulation of conventional optimization energy model • Formulation of new agent-based energy model • Analysis results • 4 case settings for calculations • Difference of results between 4 cases • Summary

18. Summary • New agent-base global energy model was developed, which allows us to treat conflicts of domestic policies • The preliminary analysis shows: • Solution of 1 agent case in new agent-based model actually corresponds to optimal solution in conventional model • As number of agents increases, energy system becomes inefficient • With sufficient agents, the energy system might achieve the efficiency again Future works: • To introduce more segmentalized agents and dynamism • To examine the realities of calculation results • To deal with carbon dioxide trading

20. Appendix

21. Ocean Transportation Paths Ground Transportation Paths Electric Transmission Paths Common Structureof Our Global Energy Models • Energy transportations between nodes

22. Case Setting for Calculation Grouping together regions into an agent Setting of Agents World Division

23. Formulation of NewAgent-based Global Energy Model • The calculation flow Reinforcement Learningby agents 1.Allocating Calculation 2. Calculate Reward 2. Calculate Reward 3. Update Value function / Error function 3. Update Value function / Error function 4. Update strategy 4. Update strategy 5. Decide Action 5. Decide Action Agent 1 Agent 2 Action Action

24. Formulation of Agent-based Model(1) Allocating calculation • Optimization calculation Utilizing conventional model

25. Formulation of Agent-based Model(2) Calculate Rewards • Calculate reward of each agent

26. Formulation of Agent-based Model(3) Update Value function / Error function • Actor-Critic Method of reinforcement learning

27. Formulation of Agent-based Model(4) Update strategy / (5) Decide Action • Actor-Critic Method of reinforcement learning 正規乱数発生 平均μ、標準偏差σの増減

28. Calculation ResultsOil Trading 2 agents 1 agent 6 agents 8 agents

29. The validity of New Agent-based Model (1) • OPEC monopolization case • Case Setting: OPEC, Others (2 agents) • Focused attention on oil trading • Only agent OPEC charges premium on its export, and agent Others has no countermeasure ⇒ This case can be reduced to a simple one-dimension search problem ⇒ We can calculate the theoretical monopoly price using conventional model

30. The validity of New Agent-based Model (1) • Learning solution is equilibrium solution in multi-agent formulation and corresponding to one of local minimum points ⇒ We can conclude that these results demonstrate that the new multi-agent based model works as designed Result of Agent-based Learning Result of one-dimension search Theoretical Best Premium 690[$/TOE] Iteration of Calculation Premium Price OPEC’s Strategy Energy System Cost Learning Solution Learning Solution 632[$/TOE] Relationsihp between OPEC’s premium and its Energy System Cost Progress of leaning strategy of OPEC