National data – local knowledge Using administrative data

1. National data – local knowledge Using administrative data David McLennan & Kate Wilkinson Social Disadvantage Research Centre Department of Social Policy and Social Work University of Oxford

2. Administrative data: a targeting, monitoring and evaluation resource

3. National Strategy for Neighbourhood Renewal • Aims to ‘narrow the gap’ between the most deprived neighbourhoods and the rest of the country of a range of key outcomes • Five priority themes identified: • Lower Worklessness • Lower Crime • Better Health • Better Skills • Better Housing the Physical Environment

4. Measuring Outcomes • Establish baseline • Pre- and Post-intervention • NDC v LA v Region v England • Time series • What is success? • Sustainability

5. Evaluation data sources Surveys + clear research focus + many valuable indicators of social deprivation - sampling error - results often not reliable at small area level - very expensive Census + near 100% coverage of entire population + results reliable at small area level - only every ten years - few suitable indicators of social deprivation - extremely expensive Administrative Data + near 100% coverage of population of interest + constantly updated + results reliable at small area level + already collected for operational purpose - some indicators are proxies - dependent upon support of data providers - data protection

6. Worklessness and Low Income

7. Worklessness & Low Income • Worklessness • Worklessness: Unemployment + Work-Limiting Illness • Unemployment: numbers and proportions of people aged 16-59 receiving Job Seekers Allowance • Work-Limiting Illness: numbers and proportions of people aged 16-59 receiving Incapacity Benefit or Severe Disablement Allowance • ‘Exit Rates’ from unemployment, illness and overall worklessness • Low Income • Proportion of adults and dependent children (aged 0-59) living in households receiving means-tested out-of-work benefits (Income Support + income-based Job Seekers Allowance)

8. Work and Pensions Longitudinal Study • Database of all spells of benefit receipt (DWP) and all tax records (HMRC) from June 1999 onwards. • Spells linked together using individual person unique reference number. • Includes details of person’s age, gender, home postcode, number of children, age of youngest child, spell type, spell start and end etc. • Over 164 million records and growing… • DWP’s primary research tool and the source for their neighbourhood statistics data

9. Worklessness example • An NDC area sees its unemployment rate change as follows: • 12% in 2001 • 10% in 2003 • 8% in 2005 while the local authority unemployment rate stays the same. • Success? • Change in unemployment rate could be due to: a) unemployed people in the NDC area moving into jobs b) unemployed people in the NDC becoming unable to work due to illness c) new people moving into the NDC area who are not unemployed

10. Health

11. Health Indicators • Standardised Mortality Ratio • A measure of the number of deaths in the NDC area compared to the expected level given the area’s age and gender structure • Standardised Illness Ratio • A measure of the prevalence of illness in the NDC area compared to the expected level given the area’s age and gender structure • Mental Illness Rate • proportion of adults under 60 suffering from mood or anxiety disorders in each area • Low Birth Weight • Percentage of single live births classed as low birth weight in a 5 year time period

12. Standardised Mortality Ratio

13. Crime

14. Crime Indicators • Violence Rate • Number of violent crimes per 1000 ‘at-risk’ population • Burglary Rate • Number of burglaries per 1000 ‘at-risk’ properties • Theft Rate • Number of thefts per 1000 ‘at-risk’ population • Criminal Damage Rate • Number of criminal damage crimes per 1000 ‘at-risk’ population • ‘Total Crime’ Rate • Number of violence, burglary, theft and criminal damage crimes per 1000 ‘at-risk’ population

15. Crime Rate Numerators • Individual level recorded crime from all 39 police forces • Crime type, date/time occurrence, date recorded, grid reference and/or postcode of occurrence • 33 different crime types under the broad headings of: • Violence crime • Burglary • Theft • Criminal Damage

19. ½ a crime to NDC area ½ a crime to non-NDC area

20. Estimates of properties / population ‘at-risk’ • At risk properties = Residential properties + Commercial/industrial properties • At risk population = Resident population (minus prison) + Workers + Students + Shoppers / recreational users + Passers by + etc…

21. Estimates of properties / population ‘at-risk’ • At risk properties = Residential properties Total Dwellings (Census) + Commercial/industrial properties + OS Address Point • At risk population = Resident population (minus prison) Resident Population (estimates) + Workers - Prison Pop + Students + Workplace Pop (Census) + Shoppers / recreational users + Passers by + etc…

22. Estimates of properties / population ‘at-risk’ • At risk properties = Residential properties Total Dwellings (Census) + Commercial/industrial properties + OS Address Point • At risk population = Resident population (minus prison) Resident Population (estimates) + Workers - Prison Pop + Students+ Workplace Pop (Census) + Shoppers / recreational users + Passers by + etc…

23. Education

24. Indicators & data sources

25. Using the education data • Example – KS4 performance 2002-2005, NDC, LA and Region • A picture of educational performance and attainment from age 11-20 to allow comparison: over time; with Government targets; with district, regional and national figures • Cohort tracking – track performance of 2002 KS2 cohort to 2005 KS3

26. Data limitations – some examples • Staying on rate and entry to higher education indicators are proxies not actual measurements • Pupil cohorts can have different characteristics across years • Education indicators may not be comparable with locally sourced statistics or national statistics from different providers: • GCSE indicator dependent on whether pupils are included who left school at 16 and were not entered for any exams • KS2 and KS3 also depend on whether or not pupils who are absent for the test are included in the denominator( for example) • DfES generally produces data at school level rather than area level – all NDC pupils do not go to the same schools…

27. Coping with data limitations • Look at outcomes across the age range – performance across the range of indicators often varies by NDC • Consider change across a long time-series – data is available from 2002-2005, looking at a longer time period is a better indicator of long-term trends • Use survey data to supplement administrative indicators • Remember that indicator definitions are consistent between areas and over time BUT other factors (i.e. pupil characteristics) can influence performance and these vary over time and between areas In Summary… • Performance across the range of education indicators varies within an NDC • Changes over time are important as well as relative performance – all areas start from a different baseline

28. NDC variation across education indicators

29. Housing

30. Indicators & data sources • Mean price of houses sold by type: flats, terraced, semi-detached, detached, Source: Land Registry • Number of houses sold by type: flats, terraced, semi-detached, detached, Source: Land Registry Uses & limitations • Comparing change in house prices over time and relative to district, region and England However…. • No information about type of house i.e. no. of bedrooms so difficult to make accurate comparisons • No information about turnover i.e. rate of house sales – will be available in the future • From current data difficult to draw conclusions about area desirability

31. Population Estimates

32. Methodology & data sources • Population counts by age and gender within 5 year age groupings from 1999-2005 • Population counts from various administrative data sources: • Child Benefit (0-14) • Patient Registration (0-90+) • Super Older Persons Database (65+) • Only Patient Registration data covers the entire age range and all data sets known to have weaknesses in particular areas and for particular age ranges • Methodology developed to test data reliability and make population estimates based on relative accuracy of each data source • ONS work on producing small area estimates used to supplement and improve our methodology

33. Uses & limitations • Population change can be an indicator of area desirability….. BUT • Estimates are estimates and may be inaccurate – they rely on the quality of the administrative data • Local authority mid-year estimates have been revised since 2001 • Populations may change as a result of housing regeneration

34. Summary

35. Administrative data indicators • Useful tool for measuring performance at NDC level because: • Data is collected routinely for operational purposes so enables a consistent time-series to be built up • They enable measurement across a variety of themes so can be linked to specific interventions • They do not suffer from sampling error so reliable at small area level • They can be easily and consistently compared over time and between areas but there are limitations….

36. Administrative data limitations • There may be differences in definitions, data sources and time collection points between local and national data • National data available from different providers and may use different definitions • Sometimes indicators are proxies or estimates of events or outcomes • Care needed attributing indicator changes to programme interventions Suggestions or comments on indicator packages and data provision are welcome and are best directed through the NDC Reference Group.