Travel time reliability and customer satisfaction in Dutch rail transport

1. Travel time reliability and customer satisfaction in Dutch rail transport Joint with: Piet Rietveld

2. 2 Background (1) In the conceptual model on this slide, reliability is seen as one of several attributes of a trip with a certain transport mode, together with costs, travel time and other attributes. The level or quality of each of these attributes determines the satisfaction with it, while the satisfaction with each attribute determines the overall satisfaction with the transport mode. This in turn affects travel behavioural decisions related to mode choice, route choice, overall demand etc. ? Now, the next questions is: how should we exactly define reliability? In the conceptual model on this slide, reliability is seen as one of several attributes of a trip with a certain transport mode, together with costs, travel time and other attributes. The level or quality of each of these attributes determines the satisfaction with it, while the satisfaction with each attribute determines the overall satisfaction with the transport mode. This in turn affects travel behavioural decisions related to mode choice, route choice, overall demand etc. ? Now, the next questions is: how should we exactly define reliability?

3. 3 Reliability measurement in the Dutch rail sector Ministry of Transport / Dutch Railways: Punctuality: �the percentage of trains that arrive with less than 3 minutes of delay� Points of critique: Focus on travel time reliability Focus on single rail trip, not on door-to-door journey Focus on punctuality on arrival, not on departure. 3 minutes is arbitrary, no attention to length of delay Focus on delays only, not on early arrivals/departures. Based on trains, not on passengers Is this critique valid ? In the Netherlands the Dutch Railway company is held accountable by the Ministry of Transport for the so-called punctuality which is defined as� � Given the ambitions of the Dutch Railways to be a customer-oriented company, this seems a rather process-oriented way of measuring reliability, which could be criticized for a number of reasons. The first critique point, which is more general , is that [�] The other critique points are more specific. ? In this study we try to investigate if these points of critique are valid and if they need to be addressed. In order to do this ?In the Netherlands the Dutch Railway company is held accountable by the Ministry of Transport for the so-called punctuality which is defined as� � Given the ambitions of the Dutch Railways to be a customer-oriented company, this seems a rather process-oriented way of measuring reliability, which could be criticized for a number of reasons. The first critique point, which is more general , is that [�] The other critique points are more specific. ? In this study we try to investigate if these points of critique are valid and if they need to be addressed. In order to do this ?

4. 4 Research objectives �we formulated two research objectives. The first is to [�], which refers to this link in the conceptual model I showed before. The second objective is to [�], which refers to this link in the model, but with different indicator and specifications of reliability. ? The outline of the presentation is as follows��we formulated two research objectives. The first is to [�], which refers to this link in the conceptual model I showed before. The second objective is to [�], which refers to this link in the model, but with different indicator and specifications of reliability. ? The outline of the presentation is as follows�

5. 5 Outline Data Part I: Importance of travel time reliability Model specification Estimation results Part II: Comparing various reliability indicators Model specification Data issues Estimation results Conclusions I will first discuss the datasets that we used. Next, in part I I will briefly discuss the model and results of the first research objective. Then, in part II, I will focus on the second research objective, in which I will discuss model, data issues and assumptions and the estimation results. And I will finish with some conclusions. ? So, I will start with the dataI will first discuss the datasets that we used. Next, in part I I will briefly discuss the model and results of the first research objective. Then, in part II, I will focus on the second research objective, in which I will discuss model, data issues and assumptions and the estimation results. And I will finish with some conclusions. ? So, I will start with the data

6. 6 Data Customer Satisfaction data (individual level) Satisfaction scores for 10 rail trip dimensions + overall satisfaction Origin-, destination-, transfer- and �home� station General individual characteristics (age, gender) Domain-specific individual characteristics (travel frequency, peak hour, car available, travel purpose, type of ticket) Travel time reliability data (station level) 6 Indicators of travel time reliability For arrival and departure For peak hours vs. off peak hours, weekend vs. weekdays On a monthly basis For this research we use two different datasets. The first dataset consists of customer satisfaction data from a questionnaire by the Dutch Railways. This dataset contains, on an individual level, satisfaction scores for 10 trip dimensions including travel time reliability as well as an overall satisfaction score. Furthermore, we have information about the O, D and T station of the individual for the trip in question and the socalled home station, which is the ususal departure station of the individual. We also have information on various individual characteristics. The second dataset consists of travel time reliability data. For each station in the Netherlands we have 6 different indicators of travel time reliability, both on arrival and departure. These data is separately available for peak hours and off peak hours, weekend and weekdays. This data is available on a monthly basis. ? For the first research objective we use only CS data.For this research we use two different datasets. The first dataset consists of customer satisfaction data from a questionnaire by the Dutch Railways. This dataset contains, on an individual level, satisfaction scores for 10 trip dimensions including travel time reliability as well as an overall satisfaction score. Furthermore, we have information about the O, D and T station of the individual for the trip in question and the socalled home station, which is the ususal departure station of the individual. We also have information on various individual characteristics. The second dataset consists of travel time reliability data. For each station in the Netherlands we have 6 different indicators of travel time reliability, both on arrival and departure. These data is separately available for peak hours and off peak hours, weekend and weekdays. This data is available on a monthly basis. ? For the first research objective we use only CS data.

7. 7 Part I: Model specification Dependent variable: Overall Satisfaction score of travelling by train Explanatory variables: Satisfaction scores dimensions 1 to 10 Control variables: Dummy variables w.r.t. individual characteristics S = a + �1S1... �10S10 +?D + � The idea in part I is to use regression analysis in which we use overall satisfaction of travelling by train as the dependent variable, while the satisfaction scores of the 10 trip dimensions are used as the explanatory variables. Furthermore, we use a number of dummy variable to control for individual characteristics. So, with this estimation model (shown on the bottom of the slide), overall satisfaction is seen as a weighted average of the satisfaction scores for each trip dimension. The coefficients are estimated and represent the weight or relative importance of each dimension. ? The estimation results are graphically presented in the following diagram,� The idea in part I is to use regression analysis in which we use overall satisfaction of travelling by train as the dependent variable, while the satisfaction scores of the 10 trip dimensions are used as the explanatory variables. Furthermore, we use a number of dummy variable to control for individual characteristics. So, with this estimation model (shown on the bottom of the slide), overall satisfaction is seen as a weighted average of the satisfaction scores for each trip dimension. The coefficients are estimated and represent the weight or relative importance of each dimension. ? The estimation results are graphically presented in the following diagram,�

8. 8 Part I: Estimation results ...which plots the estimated weights of each dimension against the average satisfaction score. The red lines represent average weight and score over all dimensions. The diagram shows that punctuality is the second most important dimension, just behind travel comfort, but unlike travel comfort, punctuality has a very low satisfaction score. The combination of.....makes that punctuality has the largest negative impact on the overall satisfaction score of the rail trip and so it appears to be the dimension that most in need of monitoring and improvement. To some degree this justifies the focus of Dutch Railways on travel time reliability. ? However, the question remains how travel time reliability should be specified and which indicators should be used to measure it. Second research objective focuses on this. ...which plots the estimated weights of each dimension against the average satisfaction score. The red lines represent average weight and score over all dimensions. The diagram shows that punctuality is the second most important dimension, just behind travel comfort, but unlike travel comfort, punctuality has a very low satisfaction score. The combination of.....makes that punctuality has the largest negative impact on the overall satisfaction score of the rail trip and so it appears to be the dimension that most in need of monitoring and improvement. To some degree this justifies the focus of Dutch Railways on travel time reliability. ? However, the question remains how travel time reliability should be specified and which indicators should be used to measure it. Second research objective focuses on this.

9. 9 Dependent variable: Satisfaction with travel time reliability Explanatory variable (6 indicators of travel time reliability; A&D): % >3 min delayed (Dutch Railways) % >9 min delayed Average delay in minutes Average delay in minutes of trains with >3 min delay 80th minus 50th percentile Standard deviation Control variables: Dummy variables w.r.t. individual characteristics SR = a + ?R+ ?D + � Part II: Model specification The idea of part II is to use regression analysis in which satisfaction with reliability is explained by various indicators of actual reliability. These indicators are: 1,2,3,4,5,6. We use dummy variables to control for individual characteristics. So the model to be estimated looks like this. ? A problem with estimation this model is related to the data.The idea of part II is to use regression analysis in which satisfaction with reliability is explained by various indicators of actual reliability. These indicators are: 1,2,3,4,5,6. We use dummy variables to control for individual characteristics. So the model to be estimated looks like this. ? A problem with estimation this model is related to the data.

10. 10 Part II: Data (revisited) Customer Satisfaction Research (individual level) Satisfaction scores for 34 trip attributes + overall satisfaction Origin-, destination-, transfer- and �home� station General individual characteristics (age, gender) Domain-specific individual characteristics (travel frequency, peak hour, car available, travel purpose, type of ticket) Travel time reliability data (station level) 6 Indicators of travel time reliability For arrival and departure For peak hours / off peak hours, weekend vs. weekdays On a monthly basis The issue is that we have the satisfaction data on the individual level and the actual reliability data on the station level. In order to link the satisfaction score of an individual to actual reliability we need information on where and when the individual travels. While we don�t have such detailed data, we do have some information about the stations the individual uses as well as information on travel frequency, car availability, travel purpose, type of ticket etc. ? Based on this information we formulate a number of assumptions... The issue is that we have the satisfaction data on the individual level and the actual reliability data on the station level. In order to link the satisfaction score of an individual to actual reliability we need information on where and when the individual travels. While we don�t have such detailed data, we do have some information about the stations the individual uses as well as information on travel frequency, car availability, travel purpose, type of ticket etc. ? Based on this information we formulate a number of assumptions...

11. 11 Part II: Assumptions Score is based on reliability at �Home� station Peak hour travellers ? reliability during peak hours Weekend card ? reliability during weekends Learning model: R = wRt + w2Rt-1+w3Rt-2���. (estimation: w = 0.4) First, for the moment, we assume that the satisfaction score of an individual depends on the R experience at the �home� station. Second, the satisfaction of peak hour travellers depends on the R experience during peak hours Third, the satisfaction of weekend card holders depends on the R experience during weekends Finally, for all travellers, we assume a learning model in which the satisfaction depends to a higher degree on more recent R experiences and to a lower degree on R experiences further in the past. ? So, using these assumptions, we estimate�First, for the moment, we assume that the satisfaction score of an individual depends on the R experience at the �home� station. Second, the satisfaction of peak hour travellers depends on the R experience during peak hours Third, the satisfaction of weekend card holders depends on the R experience during weekends Finally, for all travellers, we assume a learning model in which the satisfaction depends to a higher degree on more recent R experiences and to a lower degree on R experiences further in the past. ? So, using these assumptions, we estimate�

12. 12 Part II: Model specifications Base model: SR = a + ?R+ ?D + � Indicator: punctuality Specification 1: R = RA Specification 2: R = [wARA + wDRD], with wA + wD = 1 ...the base model presented earlier, and we estimate the model according to two different specifications. One is based on R on arrival, while in spec 2 reliability variabe is a weighted average of R on arrival and R on departure. For the second spec., the weights are estimated together with the other coefficients. ? The estimation results are shown�...the base model presented earlier, and we estimate the model according to two different specifications. One is based on R on arrival, while in spec 2 reliability variabe is a weighted average of R on arrival and R on departure. For the second spec., the weights are estimated together with the other coefficients. ? The estimation results are shown�

13. 13 �on this slide, together with some correlation coefficients we calculated between the actual punctuality and the satisfaction score. The coefficient of punctuality, for specification 1 , is -1.327, which means that an increase in punctuality from 70 to 80% leads to an increase in the satisfaction score of 0.13, which is rather low. At the bottom of the slide we see that also the model fit (R2) and the correlation coefficients show very low values. The reason for this is that, despite our data assumptions, it is difficult to directly link the satisfaction score of the individual to the actual reliability data. While this makes it difficult to draw conclusions about the precise impact of reliability on satisfaction, it is still possible to compare different specifications and indicators in their ability to explain customer satisfaction. From column 2 we see that specification 2 results in a higher coefficient, a better model fit and higher correlation values. Apparently, a specification based on both R on arrival and departure is better in explaining customer satisfaction than a specification only based on arrival. If we look at the weight estimates, we see that R on arrival has a higher weight and thus appears to be more important than R on departure. ? In order to further improve the model we make an additional assumption� �on this slide, together with some correlation coefficients we calculated between the actual punctuality and the satisfaction score. The coefficient of punctuality, for specification 1 , is -1.327, which means that an increase in punctuality from 70 to 80% leads to an increase in the satisfaction score of 0.13, which is rather low. At the bottom of the slide we see that also the model fit (R2) and the correlation coefficients show very low values. The reason for this is that, despite our data assumptions, it is difficult to directly link the satisfaction score of the individual to the actual reliability data. While this makes it difficult to draw conclusions about the precise impact of reliability on satisfaction, it is still possible to compare different specifications and indicators in their ability to explain customer satisfaction. From column 2 we see that specification 2 results in a higher coefficient, a better model fit and higher correlation values. Apparently, a specification based on both R on arrival and departure is better in explaining customer satisfaction than a specification only based on arrival. If we look at the weight estimates, we see that R on arrival has a higher weight and thus appears to be more important than R on departure. ? In order to further improve the model we make an additional assumption�

14. 14 Part II: Additional assumption If the individual has fidelity card for a specific route or travel purpose is commuting then satisfaction depends on reliability at Origin/Destination, Transfer and �Home� station. Base model: SR = a + ?R+ ?D + � Specification 3: R = WA[wA,O-DRA,O-D+wA,TRA,T+wA,HRA,H]+ WD[wD,O-DRD,O-D+wD,TRD,T+wD,HRD,H] [read assumption] We use the same base model, but a different specification for the R. R is again a weighted average of R on arrival and departure, but now both R on arrival and R on departure are each weighted averages of reliability on the O-D, the transfer and the home station. All weights are estimated together with the other coefficients. ? The estimation results of this specification�[read assumption] We use the same base model, but a different specification for the R. R is again a weighted average of R on arrival and departure, but now both R on arrival and R on departure are each weighted averages of reliability on the O-D, the transfer and the home station. All weights are estimated together with the other coefficients. ? The estimation results of this specification�

15. 15 �here in column 3 show that the impact of punctuality is much higher, and also the model fit and the correlation coefficients show higher values. �here in column 3 show that the impact of punctuality is much higher, and also the model fit and the correlation coefficients show higher values.

16. 16 The estimated weights show that R arrival is again more important than departure. If we have a more detailed look at the weights per station�.The estimated weights show that R arrival is again more important than departure. If we have a more detailed look at the weights per station�.

17. 17 �we see that in particular the arrival at the transfer station has a high weight. Apparently it is very important to arrive in time on the transfer station, which could be related to the risk of missing connections, which may result in large time-losses. With respect to reliability on departure it is not the transfer station but the other stations that have a high weight. A reason could be that a delayed departure from the other stations increases the probability of arriving late at the transfer station and therefore the risk of missing connections. In fact, a delayed departure from the transfer station decreases that risk, which could explain the fact that the weight of the departure from the transfer station is very low. ? So far we used the 3 min punctuality indicator of the Dutch Railways as a reliability variable, although we looked at different specifications, now let�s have a look how the other 5 indicators of TTR compare. �we see that in particular the arrival at the transfer station has a high weight. Apparently it is very important to arrive in time on the transfer station, which could be related to the risk of missing connections, which may result in large time-losses. With respect to reliability on departure it is not the transfer station but the other stations that have a high weight. A reason could be that a delayed departure from the other stations increases the probability of arriving late at the transfer station and therefore the risk of missing connections. In fact, a delayed departure from the transfer station decreases that risk, which could explain the fact that the weight of the departure from the transfer station is very low. ? So far we used the 3 min punctuality indicator of the Dutch Railways as a reliability variable, although we looked at different specifications, now let�s have a look how the other 5 indicators of TTR compare.

18. 18 This table shows estimation results for all 6 indicators, all estimated with specification 3. As the graph shows, in terms of model fit there are only slight differences between the indicators, but in terms of correlation between the reliability indicator and the customer satisfaction, we see that the indicators in the 1st and 3rd column, i.e. the indicator based on �% delayed > 3 min� and the one based �delay in minutes� clearly outperform the other indicators. These two indicators are the only ones that clearly outperform the punctuality indicator used by the Dutch Railways. With respect to the estimated weight it is noticeable that, for the indicators based on travel time variation, the pattern of estimated weights is rather different than for the other indicators, in the sense that the estimated weights suggest that reliability on departure is more important than on arrival. This may be explained from the fact that, unlike the other indicators, these indicators pick up negative effects of early departures. Early departures lead to potentially large time-losses due to missing trains. This is particularly true at the transfer station, which is confirmed by the fact that the estimated weight is particularly high for departures from the transfer station. ? With this final result we go to the conclusions�This table shows estimation results for all 6 indicators, all estimated with specification 3. As the graph shows, in terms of model fit there are only slight differences between the indicators, but in terms of correlation between the reliability indicator and the customer satisfaction, we see that the indicators in the 1st and 3rd column, i.e. the indicator based on �% delayed > 3 min� and the one based �delay in minutes� clearly outperform the other indicators. These two indicators are the only ones that clearly outperform the punctuality indicator used by the Dutch Railways. With respect to the estimated weight it is noticeable that, for the indicators based on travel time variation, the pattern of estimated weights is rather different than for the other indicators, in the sense that the estimated weights suggest that reliability on departure is more important than on arrival. This may be explained from the fact that, unlike the other indicators, these indicators pick up negative effects of early departures. Early departures lead to potentially large time-losses due to missing trains. This is particularly true at the transfer station, which is confirmed by the fact that the estimated weight is particularly high for departures from the transfer station. ? With this final result we go to the conclusions�

19. 19 Conclusions Ministry of Transport / Dutch Railways: Punctuality: �the percentage of trains that arrive with less than 3 minutes of delay� Focus on travel time reliability is justified Connectivity reliability is very important Reliability on departure also plays a role < 3 minutes indicator performs well but not better than indicator based on average delay in minutes Not only delays but also early departures affect satisfaction �in which I return to the points of critique on the punctuality indicator used by the Dutch Railways [1] As we showed, travel time reliability is one of the most important dimensions of the rail trip and, based on the low satisfaction score, there is some justification for this. [2] While we did not focus explicitly on other transport modes, we did show that connectivity reliability between trains is of particular importance, so the critique is valid. [3] Arrival is more important than departure, but in certain cases departure matters. [4] The indicator based on 3 minutes performs best, together with the indicator based on delay in minutes, so the use of this indicator is justified [5] From the results of the travel time variation indicators, we saw that early departure does matter in some situations [6] We did not address this in this particular analysis, although from descriptive analysis of punctuality statistics it followed that it does matter whether you weight for number of passengers or not. �in which I return to the points of critique on the punctuality indicator used by the Dutch Railways [1] As we showed, travel time reliability is one of the most important dimensions of the rail trip and, based on the low satisfaction score, there is some justification for this. [2] While we did not focus explicitly on other transport modes, we did show that connectivity reliability between trains is of particular importance, so the critique is valid. [3] Arrival is more important than departure, but in certain cases departure matters. [4] The indicator based on 3 minutes performs best, together with the indicator based on delay in minutes, so the use of this indicator is justified [5] From the results of the travel time variation indicators, we saw that early departure does matter in some situations [6] We did not address this in this particular analysis, although from descriptive analysis of punctuality statistics it followed that it does matter whether you weight for number of passengers or not.

20. 20 Thank you!mbrons@feweb.vu.nl