A1: Surveys for Behavioural Experiments

1. A1: Surveys for Behavioural Experiments Peter Jones, CTS, UCL Regine Gerike, TUD Giorgia Servente, Polito, TO

2. Workshop Context • Many countries are considering using VBC programmes to influence (travel) behaviour • How should such programmes be monitored and evaluated? • Are there methodological problems unique to VBC? • Are there best practice recommendations? • What are the methodological research challenges?

3. SURVEY DESIGN PROCESS Existing Travel and Consumption Patterns ** Need for Change: Policy OBJECTIVES? Voluntary Behaviour Change Intervention POST Travel and Consumption Patterns

4. SURVEY DESIGN PROCESS Existing Travel and Consumption Patterns ** Need for Change: Policy OBJECTIVES? CONTROL Group Voluntary Behaviour Change Intervention • What were the IMPACTS of the intervention? • On Subjects • On Others POST Travel and Consumption Patterns

5. SURVEY DESIGN PROCESS Existing Travel and Consumption Patterns ** Need for Change: Policy OBJECTIVES? CONTROL Group Voluntary Behaviour Change Intervention • What were the IMPACTS of the intervention? • On Subjects • On Others WHAT Led to the Observed Changes? = PROCESS POST Travel and Consumption Patterns

6. Key Issues - 1 What are we trying to measure? • What are our objectives? (reduce car use, save energy, increase physical activity…) • What about secondary impacts? (local economy, traffic accidents…) • Are we seeking just to measure change or understand reasons for people to/not to change?

7. Key Issues - 2 Ways of measuring behaviour: • Take up of marketing information • Reported behaviour (trips, durations, etc) • Direct individual consumption measures (odometers, GPS, etc) – reported or relayed • Aggregate area measures (bus passengers, traffic, etc)

8. Key Issues - 3 How to identify ‘real’ change: • Need for ‘control’ groups – difficult! • Potential biases in reporting, especially with VBC programmes • Retrospective vs. pre-determined • Key importance of question framing • Greater risks where respondents associate survey with VBC programme • Dangers of using panel surveys

9. Key Issues - 4 How to assess statistical significance: • Systematic vs. random errors • What level of significance do we really need? • Large samples are expensive to collect • Possible scope for meta analysis? • Importance of % response rate: • Higher % => lower mean trip rate

10. Key Issues - 5 When should impacts be measured? • Allow time for effects to settle down (e.g. after expiry of free bus tickets) • Match ‘before’ and ‘after’ seasonally How long do impacts last? • May increase over time (snowball effect)..?

11. Key Issues - 6 How can we understand causes of changes and willingness to change? • Trips as multi-dimensional packages • Need to ground in psychological theory (e.g. Theory of Planned Behaviour) • Important role for qualitative research …..All of which can help inform best type of VBC interventions

12. Some Key Research Areas • Accuracy/role of ‘meter’ readings • How best to integrate data sources • Defining effective control groups • Cultural effects (survey and behaviour) • Deeper understanding of change processes • Importance of ‘interviewer’ effects? • Greater use of experimentation (reduced need for precision in evaluating practical applications?) • ‘Tool kit’ for VBS evaluation: low cost options for developing countries