The effectiveness of healthcare regulation: lessons from research

1. The effectiveness of healthcare regulation: lessons from research 7th EPSO conference – Cork, 15 May 2009 Professor Kieran Walshe Manchester Business School kieran.walshe@mbs.ac.uk

2. Overview • Defining regulation, and the growth of regulation in healthcare • Thinking about regulatory agencies and processes: theories and frameworks • The impact of regulation: some ideas and a case study • Responsive regulation and regulation for quality improvement • Conclusions

3. Defining regulation “Sustained and focused control exercised by a public agency over activities that are valued by a community” (Selznick 1985) • Public interest purpose • Regulatory agency • Formal regulatory powers and processes • Economic and/or social regulation

4. Medicines and Healthcare Products Regulatory Agency Council for Healthcare Regulatory Excellence Nursing and Midwifery Council European Medicines Evaluations Agency General Medical Council National Institute for Clinical Excellence General Dental Council People Technologies Health Professions Council General Optical Council Equality and Human Rights Commission Health and Safety Executive Organisations Environment Agency Healthcare Commission Mental Health Act Commission Audit Commission

5. The international regulatory landscape

6. What is regulated?

7. What does it cost?

8. The growth of regulation: likely causes • Societal expectations and changing attitudes – to risk, quality and safety in healthcare • Diminished trust in health professions and organisations – the rise of the “audit society” • Evidence of poor performance - regulation as a political response to scandals and failures in care • Changing ownership and structure of healthcare industry – from state to private/corporate, from central planning and direction to market/competition

9. Questions, questions… • Does regulation work? • Why and how does it change organisations’ behaviour and performance? • What are the principles of effective regulation? • What can we learn from how regulation is done in other countries or other sectors? • Why regulate in the first place? • How should we design the “regulatory regime”?

10. More questions... • Should we set standards or rules for people to follow? About what? How detailed? How strict? • How do we decide who can do the business? Do we licence or certify people to operate? How? • What shall we inspect? How often do we need to inspect or check up on people? How do we do it? • What do we do when we find people are not performing or are breaking our rules? When should we use sanctions or punishments? • How do we show that what we are doing is achieving something (and what is it we are trying to achieve?)

11. Some theoretical perspectives • Public choice theory • Public interest or interest group theory • Institutional theory • Principal agent theory • Game theory

12. Considerations for regulatory regimes

13. Framework for analysing regulation • Policy purpose or objectives • Regulatory body or agency • Range and scope of regulated activities • Regulatory regime

14. Policy purpose or objectives • Reasons to regulate: market failure, social goals and organisational characteristics/design • Improving performance – efficiency, effectiveness at an organisational or a system level • Making regulated organisations accountable – empowering consumers, • Providing information (“sunshine regulation”) – which others use in making decisions • But – objectives conflict; must be negotiated; are one among many influences

15. Regulatory bodies or agencies: autonomy and accountability • Three basic types of agency: • Part of a government department (eg HMIC, HMIP) • An arms-length agency/NDPB (eg OFSTED, Healthcare Commission) • An autonomous statutory body (eg GMC) • What statutory powers and authority does it have? • How is it resourced – from grants/fees/levies? • How is it governed – board/management arrangements? • How and to whom is it accountable?

16. Regulated organisations and activities • Scope of regulation – vertical and horizontal • Number and size of regulated organisations • Nature of regulated organisations and their environment • Heterogeneity, and ideas of “amoral calculators” and “good hearted compliers”

17. Regulatory regime • Direction – system level actions like publishing standards or directives, setting rates/tariffs, etc • Measurement – organisational level actions like self-assessments, inspections, permits/authorisations, investigations • Enforcement – publication of information, recommendations for action, fines/penalties/sanctions, incentives and awards, interventions in management, licensing/delicensing

18. Providers’ views Regulatory “burden” and overregulation Multiple, duplicated oversight by regulators Restrictive, rule-driven regime Disconnected from improvement Public views Close and cosy relationship between regulator and providers Undemanding oversight, feeble sanctions Window-dressing for the industry and professions The impact of regulation: perceptions • Regulator’s views • Necessary, effective and focused oversight • Key driver for improvement – would not happen otherwise • Limited regulatory resources constrains what can be achieved

19. How regulation has an impact • Direction – regulator issues rules/requirements and organisations comply of their own accord • Measurement – prospect of being assessed or inspected stimulates change in organisation; results of measurement prompt reflection and change • Enforcement – recommendations from regulator, sometimes backed by sanctions or incentives • Indirect effects – other stakeholders pay attention to information from regulator in making their decisions

20. Specific changes Organisational reflection Priority to important but non-urgent Leverage for internal groups Ratchet effect of raising standards Temporary improvement not sustained Pointless conformance Defensive or minimal compliance Creative compliance Inhibiting innovation Distorting internal priorities Opportunity costs Positive and negative impacts

21. Common regulatory problems • Regulatee resistance • Ritualistic compliance • Regulatory collusion and capture • Measurement ambiguity and problems • Goal displacement • Regulatory proliferation and growth • Regulatory rigidity, juridification and inflexibility

22. Problems knowing what is effective regulation • Methodological – difficulties of measurement, causation and interpretation • Practical – regulatory agencies find it hard to experiment and innovate, are often risk averse • Cultural – regulators and other stakeholders lack an evaluative culture and aren’t necessarily interested in finding out what works • Political – purpose of regulation is both symbolic and functional, its not just about technical effectiveness

23. The impact of regulation: evidence from MARQUIS • Survey of 69 expert informants in 24 of 25 countries of the EU – focused on national policies and strategies for quality improvement • Survey of 389 hospitals across 8 countries and site visits to some hospitals, gathering data at organisation, department and clinical service level • Key conclusions – external pressure from regulation, accreditation etc linked to more developed/mature QI systems at organisational level, linked to better quality care at clinical level

24. The impact of regulation: a case study of the Commission for Health Improvement • Range, nature and appropriateness of recommendations in CHI reviews • Appropriateness, resource implications and viability of Trust action plans • Extent of implementation of action plans, barriers to progress, future expectations • Impact on trust organisation and patient care Benson LA, Boyd A, Walshe K. Learning from regulatory interventions in healthcare: The Commission for Health Improvement and its clinical governance review process. Clinical Governance: An International Journal, 2006; 11(3):213-224.

25. Research methods • Selected sample of 30 NHS trusts which underwent clinical governance review in 2001 or 2002 • Existing data: review report, action plan, StHA progress review • New data: questionnaire survey and case study interviews • Finding, categorising, tracking and following up recommendations – “key areas for action” or KAAs

26. CHI reports varied greatly in structure, content and presentation • Variation – from 20 to 80 pages, from 5 to 57 KAAs, differing structures and designs, different focus and emphasis • Causes of variation

27. Widespread acceptance of CHI’s recommendations • Generally confirmatory – NHS trusts agreed wholly or partly with 92% of KAAs in survey; 77% of issues raised before in trust • Recommendations “really came from us”. Trust had “no qualms”. Report was “fair and not surprising”. “Report … did not find anything that was previously not known”

28. Most of CHI’s recommendations focused on systems, processes and management

29. Most of CHI’s recommendations focused on systems, processes and management

30. The nature of CHI’s recommendations and how they were expressed varied greatly • Type, level and measurability of change • Clarity of purpose (what needs to be achieved) and prescription (what should be done)

31. NHS trust action plans also variable in structure, content and presentation • Variation – from 4 to 89 pages, 25 to 247 action points • Some tackled recommendations embedded in report text, others didn’t • Structure and ordering variable, hard to connect with KAAs in report, easy to miss some KAAs • Some KAAs reworded or edited

32. NHS trust action plans also variable in structure, content and presentation

33. NHS trust action plans also variable in structure, content and presentation

34. NHS trust action plans also variable in structure, content and presentation

35. Action plans mostly addressed CHI’s recommendations

36. Action plans mostly addressed CHI’s recommendations

37. Action plans mostly addressed CHI’s recommendations

38. Most of CHI’s recommendations have been acted upon

39. Most of CHI’s recommendations have been acted upon

40. Most of CHI’s recommendations have been acted upon

41. Most of CHI’s recommendations have been acted upon

42. Most of CHI’s recommendations have been acted upon

43. Most of CHI’s recommendations have been acted upon

44. Correlated: Measurability of KAA Whether action plan addresses KAA Timescale for action plan Clinical governance review area Trust NHS trust star rating Not correlated: Whether KAA is a priority Depth, breadth, type of change No of action points Clarity of timescale There are some important factors involved in securing implementation

45. There are some important factors involved in securing implementation

46. There are some important factors involved in securing implementation • “But where they scored (meaning CHI) was things like they said we needed to do a staffing review, in fact, we had done it before we had the CHI review, but they brought it up in the report which enabled us to use it in our SAFF negotiations to get further investment in staffing. So we used it again as a lever for change but also a lever within the SAFF process…” • “What made a big difference in this organisation was seeing it in black and white in terms of lack of leadership and strategic direction and again once the execs. and board had been on that (development) programme they had a better understanding of what was expected of them and in terms of what the organisation had to do as well. I think that has made a big difference in the review of arrangements for clinical governance in the organisation…”

47. There are some important factors involved in securing implementation • “Some new things for the Trust certainly,. The public involvement stuff… without the focus of the CGR they would not have made much progress with that. The concerns about Cardiology were raised by the Deanery before ……but the review did act as a catalyst to the patient and public stuff in particular.” • “The review and report helped us to win over doubting clinicians, because it was not a “tick box” exercise. This was the biggest impact. Consultants who thought it was a passing fad now realise that CHI, CNST etc are part of the regulatory framework, so we/they need to be consistent about clinical governance – it is fundamental.”

48. Policy implications from the study • Invest more regulatory attention and effort across the regulatory cycle • Make regulatory interventions more consistent in approach • Ensure all recommendations are constructed, expressed and communicated clearly • Require detailed, explicit and comprehensive action plans, focus followup on the original review • Shorten the regulatory cycle of attention, intervention and action

49. Responsive regulation • Regulatory regime recognises and responds to the diversity of regulated organisations • Regulatory interactions and interventions are highly contingent on how individual organisations behave • “One size fits all” approaches are eschewed in favour of more flexible and graduated approaches • Regulators and regulatory staff have considerable freedom to exercise judgement (Ayres and Braithwaite 1992)

50. Responsive regulation (Reiss 1984)