Designing and aggregating experts for energy demand forecasting

1. Designing and aggregating experts for energy demand forecasting Yannig Goude EDF R&D Georges Oppenheim UPEM & Paris 11 Pierre Gaillard EDF R&D, HEC Paris-CNRS Gilles Stoltz HEC Paris-CNRS SESO 2014 International Thematic Week “Smart Energy and Stochastic Optimization'' June 23 to 27, 2014

2. Industrial challenges • Smart grids • More and more « real time » data (ex: linky, 1million meters in 2016) • Demand response (new tariffs, real time pricing…) • New communication tools with customers (webservice, on-line reporting….) • Renewables energy development • A more and more probabilistic context • Opening of the electricity market: • Losses/gains of customers • Sensors data: • Production/consumption sites • Smart home, internet of things • New usages/tariffs: • Electric cars • Heat pumps, smart phones, battery charge, computers, flat screens…. • Demand response, special tariffs (time varying…)

3. StatisTIcal challenges • Large scale data sets • Parallelizing statistical algorithms • Complex data analysing: heteregonous spatial/temporal sampling, different sources/nature of data • Adaptivity • Non-parametric models, fonctional data analysis • Model selection, data driven penalty… • Sequential estimation • Break detection • On-line update, sequential data treatment (data flow, connection to big data) • Aggregation with on-line weigths • Multi-scale models • Multi-horizon models • Multi level data on the grid • Data mining of time series • Large scale simulations • Simulation platform, parallel processing

4. CONTRIBUTIONS • Large scale data sets • GAM parallel processing • EDF R&D/IBM simulation platform • Adaptivity • GAM models, automatic GAM selection • functional data analysis (CLR: curve linear regression, KWF: kernel wavelet fonctional) • Sequential learning: • Adaptive GAM • Combining forecasts • Spatio temporal/multi-scale models, complex data • « Downscaling » electricity consumption: link INSEE (socio-demographic, census) data to local electricity consumption (meters, grid data) and meteo data • EDF R&D/IBM simulation platform

5. Load FORECASTING • Electricity consumption is the main entry for optimizing the production units

6. ElEctricity CONSUMPTION Data • Trend • Yearly, Weekly, Daily cycles

7. ElEctricity CONSUMPTION Data • Meteorological events • Special days

8. GAM (GENERALIZED ADDITIVE MODELS) • A good trade-off complexity/adaptivity • Publications • Application on load forecasting • A. Pierrot and Y. Goude, Short-Term Electricity Load Forecasting With Generalized Additive Models Proceedings of ISAP power, pp 593-600, 2011. • R. Nédellec, J. Cugliari and Y. Goude, GEFCom2012: Electricity Load Forecasting and Backcasting with Semi-Parametric Models, International Journal of Forecasting , 2014, 30, 375 - 381. • GAM « parallel »: BAM (Big Additive Models) • S.N. Wood, Goude, Y. and S. Shaw, Generalized additive models for large datasets, Journal of Royal Statistical Society-C, 2014. • Adaptive GAM (forgetting factor) • A. Ba, M. Sinn, Y. Goude and P. Pompey, Adaptive Learning of Smoothing Functions: Application to Electricity Load Forecasting Advances in Neural Information Processing Systems 25, 2012, 2519-2527.

9. GAM

10. GEFCOM competition • 20 substations on the US grid • 11 temperatureseries • hourly data fromjanuary 2004 to june 2008 • 9 weeks to predict : 8 from 2005 to 2006, and the one following the train set (no temperatureforecastavailable) • 105 teams ? One issue : no localisation information http://www.kaggle.com/c/global-energy-forecasting-competition-2012-load-forecasting

11. GEFCOM competition Nedellec, R.; Cugliari, J. & Goude, Y.GEFCom2012: Electric load forecasting and backcasting with semi-parametric modelsInternational Journal of Forecasting , 2014, 30, 375 - 381

12. GEFCOM competition

13. CURVE LINEAR REGRESSION • Regressing curves on curves • Dimension reduction, SVD of cov(Y,X) , selection with penalised model selection • Scale to big data sets (SVD+linear regression) • Publications • Application on electricity load forecasting • H. Cho, Y. Goude, X. Brossat & Q. Yao, Modeling and Forecasting Daily Electricity Load Curves: A Hybrid Approach Journal of the American Statistical Association, 2013, 108, 7-21. • Cho, H.; Goude, Y.; Brossat, X. & Yao, Q, Modelling and forecasting daily electricity load using curve linear regressionsubmitted to Lecture Notes in Statistics: Modeling and Stochastic Learning for Forecasting in High Dimension. • Clusturing functional data • H. Cho, Y. Goude, X. Brossat & Q. Yao, Clusturing for curve linear regression, technical report, 2013.

14. CURVE LINEAR REGRESSION

15. Othermodels • Random forest: a popular machine learning method for classification/regression • Breiman, L., . Random Forests, Machine Learning, 45 (1), 2001. • KWF (Kernel Wavelet Functional): another approach for functional data forecasts • See: Antoniadis, A., Brossat, X., Cugliari, J., Poggi, J., Clustering functional data using wavelets. In: Proceedings of the Nineteenth International Conference on Computational Statistics(COMPSTAT), 2010. • Antoniadis, A., Paparoditis, E., Sapatinas, T., A functional wavelet–kernel approach for time series prediction. Journal of the Royal Statistical Society: Series B 68(5), 837–857, 2006. yes Bank Holiday <6°C no Temperature http://luc.devroye.org/BRUCE/brucepics.html <55GW >6°C Lag Load >55GW

16. SeQUENTIAL AGREGATION OF EXPERTS

17. SeQUENTIAL AGREGATION OF EXPERTS

18. SeQUENTIAL AGREGATION OF EXPERTS Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press (2006)

19. ExponentiallyWeightedAverage FORECASTER (EWA)

20. ExponentiaTED GRADIENT FORECASTER (EG)

21. OTHER ALGORITHMS Theoretical calibration Works well in practice Gaillard, P., Stoltz, G., van Erven, T.: A second-order bound with excess losses (2014). ArXiv:1402.2044

22. AppliCATIOn on LOAD forecasting • initial « heterogenous » experts: • GAM • KernelWaveletFunctional • CurveLinearRegression • Random Forest • Designing a set of experts from the original ones: 4 « home made » tricks • Bagging:60 experts • Boosting:Boosting: trained on suchthatperformswell45 experts • Specializing: focus on cold/warm days, someperiods of the year… 24 experts • Time scaling: MD with GAM, ST with the 3 initial experts • Publications • M. Devaine, P. Gaillard, Y. Goude & G. Stoltz, Forecasting electricity consumption by aggregating specialized experts - A review of the sequential aggregation of specialized experts, with an application to Slovakian and French country-wide one-day-ahead (half-)hourly predictions Machine Learning, 2013, 90, 231-260. • Gaillard, P. & Goude, Y., Forecasting electricity consumption by aggregating experts; how to design a good set of experts to appear in Lecture Notes in Statistics: Modeling and Stochastic Learning for Forecasting in High Dimension, 2013.

23. Combining FORECASTS combining Designing experts

24. Another Data SET: Heat DEMAND

25. PErspectives • Forecasting methods: • Industrial implementation on the way (national, substations, cogeneration central in Poland: 30% better with GAM) • CLR: improve automatic clusturing, forecasting the clusters (HMM) • Combining: • publication of the R package OPERA (Online Prediction through ExpeRts Aggregation) coming soon • application on other data sets • derive probabilistic forecasts from a set of experts • Probabilistic forecasts