Impact of Proposed Changes How to Measure and Evaluate change

1. Lecture -5 Impact of Proposed ChangesHow to Measure and Evaluate change

2. Types of Change within the Organisation Relocation of Services Referral System Skill Mix Working conditions Management Resource Allocation Access & Entitlement Staffing Levels Professional Roles & Accountability Monitoring & Reviewing Systems

3. Potential Impact of Organisational Changes The impact of change is often seen in waves that follow the initial disruption, producing both issues and opportunities.

4. Potential Impact of Organisational Changes The degree to which change impacts an organisation is dependent on how well the organisation is able to adjust itself to the changes it is experiencing (e.g. changes in technology, environment and resources) based on the change management process in place in the organisation.

5. Potential Impact of Organisational Changes Change Impacts an organization in terms of: • Organisational efficiency & effectiveness • Cost • Response Time • Staff satisfaction • Customer Satisfaction • Access • Environment

6. Organisational Efficiency & Effectiveness Lon Roberts (1994: 19) defines efficiency as "to the degree of economy with which the process consumes resources-especially time and money," while he distinguishes effectiveness as "how well the process actually accomplishes its intended purpose, here again from the customer's point of view."

7. Organisational Efficiency & Effectiveness Let's consider a surgery example. A surgeon is very skilled, perhaps the best in the country. The impending job is to operate on the patient's left knee. However, the surgeon doesn't perform all the steps of the process leading up to the surgery. Someone else marks the right knee for surgery.

8. Organisational Efficiency & Effectiveness However skilled this surgeon is, however fast he performs the surgery (i.e., however efficient he is), this process will not be effective. When the patient awakens from the surgery, he will not be a happy camper. Who will pay for a surgery performed on the wrong knee?

9. Organisational Efficiency & Effectiveness Organisational efficiency relates to the use of resources in achieving organisational goals. It refers to the amount of resources an organization uses in order to produce a unit of output. An efficient organisation is one that maximises its resources in production without waste.

10. Organisational Efficiency & Effectiveness Organisational effectiveness is the degree to which an organization achieves its goals. Organisational efficiency differs from Organisational effectiveness Changes when introduced into an organisation, impacts the organisations efficiency and effectiveness – improving it or reducing it. The organization has to be both effective and efficient to be successful.

11. Cost and Response Time Cost and time over run are frequently affected factors in change processes. Cost and Response time are two independent factors/dimensions. However they show a strong association and consequently, we will be considering them together.

12. Cost and Response Time Change (incremental, transitional or transformational) is expensive and time consuming. It is often difficult to calculate the direct cost of a change management project. ‘If you cannot understand something you cannot measure it. If you cannot measure it you cannot control it. If you cannot control it you cannot improve it’ (Sutherland)

13. Cost and Response Time Radical change requires considerable time and resources as against incremental changes. A slow and delayed change management project easily looses steam. Whereas, a too fast paced change process overwhelms all components of organizational change. Kotter (Kotter 2007) has strongly recommended that leaders should create a sense of urgency. It's always a trade- off between a worse now better than future. It is a ropewalk.

14. Staff Satisfaction Organisational changes such as relocation of services, etc can lead to considerable changes in traditional roles and responsibilities. This may represent psychosocial challenges for the employees, in the shape of • job uncertainty • the feeling that one is not mastering the new work situation or lacks the competence to do so and • consequently difficulties in implementing change processes and using new work processes.

15. Staff Satisfaction The complexity of these challenges brought about by changes lead to reduced levels of staff satisfaction and morale which may cascade into reduced productivity and efficiency. Undesirable health consequences have been known to arise on both an individual and an organisational level.

16. Staff Satisfaction All changes in an organisation involve learning. Some people thrive on new situations; others prefer to plan and prepare in advance. Preparing for change implementation reduces its negative impact on staff and staff satisfaction.  

17. Staff Satisfaction

18. Customer Satisfaction The impact of proposed change on customer satisfaction is an unintended consequence/outcome of an intended strategy/action (change implementation)

19. Staff vs Customer Satisfaction Lesson Aim Lesson Outcome Review the impact of organisational change on service users, service provision and care staff. measure impact of organizational change against agreed criteria

20. Staff vs Customer Satisfaction • There has been considerable research into the links between customer satisfaction and employee satisfaction - the argument is a very obvious one and a strong relationship has frequently being stated. • The argument - Happy employees work harder and try harder and so create satisfied customers.

21. Staff vs Customer Satisfaction • A co-ordinated customer satisfaction programme should consider linking with an employee attitude survey. • The employee attitude survey could also be used to check out how well staff believe they are satisfying customers as there could be a dangerous gap between internal perceptions of performance and those of customers • How staff treat customers is of crucial importance in achieving customer satisfaction

22. Measurement of Customer Satisfaction The 6 parts of a Customer satisfaction survey Who should be interviewed? What should be measured? How should the interview be carried out? How should satisfaction be measured? What do the measurements mean? How to use customer satisfaction surveys to greatest effect?

23. Who should be interviewed? • Key decision maker • First in line customer • Lapsed customers • Potential customers

24. What Should be Measured? In customer satisfaction surveys, we seek the views of respondents on a variety of issues that will show how the company is performing and how it can improve. We have high level questions and specific/detailed level questions Working out what questions to ask at a detailed level means seeing the world from a customers’ point of view. What do they consider important?

25. How Should the Interview be Carried Out? • Postal Surveys • Face to face interviews • Telephone Interviews Advantages, disadvantages and typical applications Various factors affect the choice of survey tool

26. How Should Satisfaction be Measured? • Open ended questions • Ratings scale Customer expectations is key and should also be measured.

27. What do the Measurements Mean? The scores that are achieved in customer satisfaction surveys are used to create a customer satisfaction index or CSI. There is no single definition of what comprises a customer satisfaction index. Some use only the rating given to overall performance. Some use an average of the two key measurements - overall performance and the intention to re-buy (an indication of loyalty) yet others may bring together a wider basket of issues to form a CSI.

28. What do the Measurements Mean? The average or mean score of satisfaction given to each attribute provides a league table of strengths and weaknesses. As a guide, the following interpretation can be made of scores from many different customer satisfaction surveys: • Means scores of over 8 out of 10 - Market leader, excellent supplier • Mean scores of 7 to 8 out of 10 - Adequate but needs attention

29. What do the Measurements Mean?Customer satisfaction and the effect on customer loyalty

30. What do the Measurements Mean?ImprovingCustomer satisfaction

31. How to Use Customer Satisfaction Survey Effect No company can truly satisfy its customers unless top management is fully behind the programme. Customer satisfaction studies are not always genuinely customer orientated e.g. Northwest Airlines & Continental - two companies, both ostensibly committed to customer satisfaction, but one markedly outperforming the other.

32. How to Use Customer Satisfaction Survey Effect A customer satisfaction index is a snapshot at a point in time. People’s views change continuously and the performance of companies in delivering customer satisfaction is also changing. Measuring satisfaction must be a continuous process.

33. Developing an Action Plan A 5 step process can be used: • Spot the gap • Challenge and redefine the segmentation • Challenge and redefine the CVP • Create an action plan • Measurement and Improvement

34. GROUP WORK/DISCUSSION

35. Evaluate Research Surveys Lesson Aim Lesson Outcome Review the impact of organisational change on service users, service provision and care staff. devise strategies and criteria for reviewing changes

36. Basics of Survey Research descriptive research Survey questionnaire successive samples causal explanation prediction Panel study evaluation Single sample

37. What do Surveys of People Measure Uses of Surveys Descriptive Research Causal Explanation Evaluation Prediction

38. Types of Survey Design Single cross section Designs for assessing change

39. Types of Data Collected in Surveys • attitudes - what people say they want • opinions - what people think might be true • beliefs - what people know is true • behaviour - what people actually do • what people are (demographic characteristics) • preferences - what people would choose

40. 7 Basic Survey Questions • Open response • Partially open response • Closed response • Semantic differential scales • Agreement and rating scales • Ranking scales • Checklists

41. Choosing a Questionnaire Format Surveys People respond to Surveys for a number of reasons: • Rewards • Cost • Trust

42. Elements of Item Construction • Writing questions • Forced choice vs non-forced choice • Response styles and response sets • Length of the questionnaire

43. Assessing Reliability and Validity Types of Validity • Face Validity • Content Validity • Concurrent Validity • Discriminant Validity • Predictive Validity • Construct Validity

44. Assessing Reliability and Validity Types of Reliability • Test-retest reliability • Internal consistency • Stability

45. Sampling Theory & Practice • Sampling Theory & Sample Section • A sample statistic • Identifying the target population • Census or sample? • Obtaining and ample sample • Obtaining a manageable collection of objects to study • Providing a qualitative representation of population characteristics

46. Sampling Theory & Practice • Providing quantitative estimates of population characteristics • Controlling for extraneous factors • Intact groups • Methods for determining sample size • Sample size in survey research • Sample size in hypothesis testing

47. Coding System and Data organisation

48. Formatting & Pilot-testing the Survey • The format of the survey • Why do a pilot-test? • Selecting the pilot test sample • Information to be collected • Participant debriefing

49. Analyzing the Data & Results • Descriptive Analysis • Presenting the results • Taking a step beyond the conclusion • Making recommendations • The report format • Paying attention to detail

50. Resource Materials • Polland Ronald Jay, A WORKBOOK FOR COMMUNITY RESEARCHERS, 1998 http://www.tfn.net/~polland/qbook.html