Neural Networks to Model Energy Commodity Price Dynamics

1. Neural Networks to Model Energy Commodity Price Dynamics M.Panella1, F.Barcellona2, V.Santucci1, R.L.D’Ecclesia2 massimo.panella@uniroma1.it francesco.barcellona@uniroma1.it rita.decclesia@uniroma1.it 1Dpt. DIET - Dpt. of Information Engineering, Electronics and Telecommunications 2Dpt. DAES – Dpt. of Economic and Social Analysis 30th USAEE/IAEE NORTH AMERICAN CONFERENCE OCT. 9-12 - 2011 Prof. M.Panella

2. Outline • Aim • Some literautre • Datasets • Experimental setup • Numericalresults • Conclusions and future developments

3. Motivation • To provide a powerful tool to replicate economic and financial series dynamics • To apply a new methodology based on a neural network approach to build a forecast for specific standard price time series of energy commodities • To backtest the results and set up risk management strategies

4. Available methodologies Forecasting process like a black box. Methodological approaches to analize time series: • Standard approaches • Modern approaches • Autoregressive models (AR) • Moving average models (MA) • ARMA models • Qualitative techniques: • opinion based approaches • testing based approaches • Quantitative techniques: • explanatory methods • time series based methods • ARIMA models • ARCH and GARCH models

5. Some Recent Literature • A.G. Malliaris e Mary Malliaris ‘Time series and Neural Networks comparison on gold, oil and the euro’, International Joint Conference on Neural Networks, 2009. • Cristina Bencivenga, Rita D’Ecclesia, ‘Integration of Energy commodity markets: the case of Europe and US’, Journal of Risk Management in Financial Institutions, Henry Stewart Publications, 2011 • Cristina Bencivenga, Rita D’Ecclesia, ‘Energy commodities price dynamics: understanding oil price volatility’, Journal of Energy Economics, 2011 • R. Gencay and M. Qi. ‘Pricing and Hedging Derivative Securities with neural Networks: Bayesian Regularization, Early Stopping, and Bagging’. IEEE 2001. • R. Garcia, R. Gencay. ‘Pricing and hedging Derivative securities with neural networks and a homogeneity hint’. Journal of Econometrics 94 (2000). • F. Malik, M. Nasereddin. ‘Forecasting output using oil prices: A cascaded artificial neural network approch’. Journal of Econometrics and Business 58 (2006).

6. Forecasting and function approximation • Given a sampled sequence S(n), the general approach to its prediction is based on the solution of a suitable function approximation problem y=f(x) Embedding technique • Problem: when dealing with real-world (often chaotic)data sequences, the previous AR predictor would hardly fail since it is based on a linear approximation model and on a trivial embedding technique.

7. Embedding of chaotic time series • The sampled sequence S(n) to be predicted originates from an unknown autonomous context: X(n+1) = F(X(n)) S(n) = G(X(n)) X(n) = State vector Embedding theory S(n) Reconstructed state-space: T: time lag between samples (AMI) N: embedding dimension (FNN) Predictor: Function approximation: (training set)

8. Radial Basis Function Neural Network F1(x) M  y = w Fj(x) Fj(x): nonlinear (vector) function j w j 1 = 1 x1 F2(x) w 2 x2  w 3 F3(x) x3 ... w M ... x D A hidden layer of radial kernels: Performs a non-linear transformation of input space. The resulting hidden space is typically of higher dimensionality than the input space FM(x) An output layer of linear neurons: The output layer performs linear regression to predict the desired targets

9. Network model in the input space (x)    x 1  2 x … … … D x Least–Squares Approximation (LSE): Mixture of Gaussian (MoG) Neural Network Parameters related to the j-th Gaussian component p(x,y | j) in the joint input-output space:

10. Normalization Fuzzification (1)(x) (1)(x) Rule 1 antecedent X y(x) x + (M)(x) (M)(x) Rule M antecedent X y(1)(x) Rule 1 consequent y(M)(x) Rule M consequent ANFIS (Adaptive Neuro-Fuzzy Inference System) Network Architecture • Sugeno 1st-order type rules: • Approximate reasoning: • Fuzzification of crisp inputs: x=[x1 x2 … xn]; • Membership functions: (k)(x)  reliability of linguistic (fuzzy) rules • Soft (fuzzy) decision:

11. Data set We used logarithmic transformed prices of both European and American energy commodities (oil, electricity and gas): a • Input-output series: • Input series (training phase): 8 two-year period from 2001-2002 to 2008-2009; • Output series (testing phase): 8 years from 2003 to 2010 (each follows the previous two years) • Unique time series for both phases (training and testing), described by data of three-year period (from 2001-2002-2003 to 2008-2009-2010). EUR • Brent • European Energy Exchange (EEX) • National Balancing Point (NBP) US • West Texas Intermediate (WTI) • PJM Interconnection (Eastern Interconnection) • Henry Hub (HH)

12. Test 1/3 The considered models are: • GARCH/ARIMA111 models using different numbers of forecasting factors and starting sample size • LSE (Least Squares Estimation) • RBF (Radial Basis Function) • MOG: developed by the Dept. DIET (gaussian model) • ANFIS2: matlab plug-in

13. Test 2/3 For eachpriceseries the robustness of the used model hasbeentested by settingdifferentvalues of the parameters: RBF: alfa (maximum percentage of neurons in the network); MOG: alfa (maximumnumberof clusters) lambda (modelweights) ANFIS2: T_OPT[1] (number of epochs in the training phase) The training process should be implemeted by firstly using an embedding technique in order to choose T and N (delay and embedding dimension) • Set as default T=1 and N=5

14. Tests 3/3 As qualitative results we consider: • Plots of the two series, real and predicted • Values that reflect the prediction reliability The prediction accuracy is measured by: • Mean Squared Error - MSE • Normalised Mean Square Error - NMSE • Signal-to-noise ratio or SNR (dB) • Mean absolute percentage error - MAPE (%)

15. Results analysis 1/5 In the testingphase of theoreticalmodels, itis essential to define some criteria to assess the efficacy of the chosen model. Three different criteria, based on a comparison of some series characteristics, are used: • Reproduce the same characteristics of trend, seasonality, volatility and jumps • Easy calculation and numerical implementation • Match of statistical properties of the two series • Summarized in the fourmoments: • Mean • StdDv • Skewness • Kurtosis RELIABLE FORECAST Features of the series probability distribution

16. Results analysis 2/5 Here are graph and table resulting from the 2003 forecast for European oil (Brent) using MOG predictor (alpha = 0.9, lambda = 0.5)

17. Results analysis 3/5 Here aregraph andtable resulting from the 2003 forecast for American oil (WTI) using MOG predictor (alfa=0.9, lambda=0.5)

18. Results analysis 4/5 Here are graph andtable resulting from the 2003 forecast for Brent using linear model (LSE) whose statistical analysis returned a perfect coincidence of all 4 moments. Perfect match

19. Results analysis 5/5 Here are graph and table resulting from the 2003 forecast for Brent using RBF model (alfa=0.5), whose statistical analysis return a good match only for first and second moments.

20. Conclusions and Future developments The accurate prediction of energy price series will allow to estimate the existing relationships between the various Commodities and to price derivative instruments in order to set up risk management strategies Structuring data: • Differentenergyassets • reference to current (actuality) • Extendforecasttimerange Modeling: • Differentforecastingmethods: • embedding techniques • subband (spectral) decomposition • Other financial tools Reduction of computational cost Correlation between NN and moments up to fourth order. This introduces new forecasting assumptions for energy commodities