Analytics at the Electricity Authority

1. Analytics at the Electricity Authority Presentation at the EPOC Winter Workshop 5 September 2013

2. Outline • Market Analytics within Market Performance • Priority projects for Market Analytics • Other significant initiatives • EMI-tools in a browser • Retail tariff information • Profile expansion model (PEM) • GEMdata update

3. Market performance – two teams Market Performance Bruce Smith, GM Market Monitoring Doug Watt, Manager Market Analytics Phil Bishop, Manager Enquiries, reviews and investigations (ERI) Provision of tools and data to analyse, monitor and report on market performance Monitor and report on market performance Workstream support Data warehouse Annual publications e.g. Year in review Data and reporting portal CBAs for Code amendments EMI-tools – develop and maintain suite of models Ad-hoc data wrangling and visualisation tools Post-implementation reviews

4. Market Analytics – near term priorities • Consolidate/rationalise/harmonise data collections in data warehouse • Access arrangements and data dictionary • Data integrity and checking procedures • Remove duplication • Upgrade data and reporting portal • Portal will render CDS redundant • More performance metrics – not just data • Hydrology dataset – more on this in a moment • Use of Github – https://github.com/ElectricityAuthority

5. More on near term priorities… • New version of EMI-tools with Doasa and vSPD • Sophisticated model code and data updating/management functionality • Improved override (vSPD) functionality – sets and parameters • Doasa more-or-less ready to go now • Doasa ‘lite’, fixed NZ network, independent inflows, etc • vSPD v1.3 available now with original EMI • Audited in same way as SPD • 3 user interfaces – EMI, Excel, standalone • vSPD v1.4 being worked on as we speak • Incorporates FTR-rent and scarity pricing rules • Will need to be audited • Codes already on Github

6. Hydrological datasets • Last update in 2010 • Update overdue and will miss the October 2013 CDS • Two main hydrological datasets: • Comit Hydro (NIWA) • Opus • About to start review aimed at: • Improving accuracy and confidence in data • HRCs, customer compensation scheme • Improve alignment between datasets around plant factors used to convert volumetric data to GWh and consented storage • Publish annually • Consolidate formats • Snow pack? Southern Oscillation Index?

7. EMI-tools in a browser • Looking at option to host EMI-tools on web server • Alongside or integrated with upgraded portal • Addresses long-standing concern that cost of a GAMS license is prohibitive for some participants and interested stakeholders • Initially just exploration and visualisation of final pricing cases, i.e. vSPD inputs and outputs, and probably Doasa too • No need for GAMS licensing to be resolved but also no ability to solve GAMS-based models • .NET framework identical beneath the user-interface layer • EMI-tools would be able to access data warehouse directly via the portal • SQL ↔ XML – by-pass GDX if just exploration and visualisation • No need for users to manage/store large vSPD input data library • No need to solve vSPD in order to get outputs – they’re in the data warehouse

8. Retail tariff information • Item C8 in Authority work programme • A major undertaking – planning just getting underway now • Most likely begin with an Auckland pilot before rolling out nationally • Will involve consultation and almost certainly some Code changes • Aim to: • Enhance analysis and performance monitoring of retail competition • Facilitate innovation in product offerings, switching providers and new entry into retail sector • Be future-proofed re: smart metering data provision

9. Profile expansion model (PEM) • A model to help understand the cost imposed on the system by various types of load • Residential versus industrial versus commercial • Flat versus peaky – day, season, year • Winter versus summer • Actual or contrived • A bit like GEM but without the computational burden • A mix of generation technologies each with capital and operating costs, operational capabilities, etc • No integer variables • Load able to be represented chronologically with up to 17,520 periods or as a load duration curve • Start with a clean sheet of paper or expand the existing generation fleet

10. Load profile factors, 2012 South Island North Island Commercial Industrial Residential

11. GEM • Generation Expansion Model • MBIE and Transpower use GEM frequently • Used much less so these days at the Authority • Several other occasional users – based on calls for help • Also, many people find the dataset useful, if not the model • About to start a small project to update the GEM database and the data preparation tools • (Probably) a Python-based toolkit to source, document, and compile GEM datasets • GDX ↔ Text/CSV ↔ XML ↔ SQL • Move codes from Google Codes to Github • Integrate into EMI-tools (next year?)