Stephen J. Porth, Ph.D. for George P. Sillup, Ph.D., M.S. CJBE Conference July 28-30, 2006

1. A Qualitative Exploration of the Relationship between Performance Appraisal and the Development of Inspired Business Leaders Stephen J. Porth, Ph.D. for George P. Sillup, Ph.D., M.S. CJBE Conference July 28-30, 2006

2. Key Findings • PA seen as a way to gain recognition for performance and as a link to becoming an IBL • Experienced (mean 16.6 years) respondents • Were successful under their PA System - “exceeded” their objectives • Some problems with bias - none as raters but two when being rated • Not enough time to implement • Vast mast majority (15 of 16 respondents) indicated that they did not have enough time to implement the PA system • No formal training • None of the respondents received formal training about the PA systems they were using • Must go beyond objective measure to develop IBLs • PA System must also introduce the employee to aspects of performance that go beyond the objective measures evaluated in results-oriented PA systems Sillup/Porth - 2

3. Research Questions • Does implementation of an effective Performance Appraisal (PA) system help to develop Inspired Business Leasers (IBLs)? • What are the key impediments to using a PA system? This includes impediments before and/or during implementation? • How can implementation be improved? • What are the implications of this research for the development of IBLs? The author wishes to acknowledge the financial support from the Saint Joseph’s University Pedro Arrupe Center for Business Ethics and the intellectual guidance from John McCall and Stephen J. Porth Sillup/Porth - 3

4. What Is Performance Appraisal (PA)? • Systematic description of the job-relevant strengths and weaknesses of an employee or an employee team • Combines two processes, observation and judgment, to assess job performance on the basis of objective and subjective indices • Objective indices are appealing because they measure a quantifiable endpoint but often measure factors beyond an employee’s control or outcomes of behavior (e.g., number of units sold) • Subjective indices measure the job performance behavior (e.g., what did it take to sell the units) Sillup/Porth - 4

5. PA and Its Influence on IBLs • PA can improve employees’ work performance by helping them to realize their full potential as IBLs • PA accomplishes this by providing information to employees and managers for use in making work-related decisions • Because PA is an inexact human process, it may only effectively evaluate how employees meet objective performance measures • e.g., assessing the number of units sold • PA may fail to evaluate attributes that lead to improved performance • e.g., integrity or other attributes especially important to the development of IBLs Sillup/Porth - 5

6. For a Positive Influence on IBLs • PA must take steps to ensure preparation for implementation and implementation addresses the following areas: • Setting fair performance standards • Meeting requirements of effective PA systems • Assessing the types of PA systems/determining which one to use • Identifying who should rate performance • Overcoming judgmental biases in rating • Establishing training programs for raters • Determining how often PA should be done • Understanding how PA fits with Total Quality Management Sillup/Porth - 6

7. Research Method • Structured interview over 30-45 minutes to assess how the shortcomings of PA potentially impact the development of IBLs • 10 questions - 9 designated areas and 1 open-ended question: • Does your PA system have clear PS? • How well does your PA system meet the requirements of an effective PA system? • What type of PA system does you company use? • Who rates PA? • Are you aware of any biases in rating? • Have you had “rater” training? When? • How often is PA conducted? • Is your company doing some variation of TQM? If so, how does PA fit with TQM? • Have you encountered any barriers to PA? • Are there any other factors that help/hinder implementation of PA? Full research instrument available in back-up slides Sillup/Porth - 7

8. Results – Profile of Respondents Sillup/Porth - 8

9. Analysis of Structured Interviews • Revealed use of results-oriented PA systems on an annual basis • Similar responses from the four companies for both the male and female respondents • Summary on table in next slide • Top-down support was identified as most important for the successful implementation of the PA system • No matter the department, all respondents had their PA conducted by their immediate managers and used multi-rater input (360-feedback) • 10 of the 16 used self-appraisal as preparation for PA • 2 different responses • Wyeth for TQM because they were the only company with a respondent from Manufacturing, the only department implementing TQM • Aetna where four customers were asked for feedback. Sillup/Porth - 9

10. Results – Frequency of Responses Sillup/Porth - 10

11. Results Summary • Taken as a whole, the interviews were positive • Regarding PA as a way to gain recognition for performance • Suggesting it as a link to becoming an IBL • None of the respondents gave me their individual PA rating but they said they “exceeded” the annual objectives • None of the respondents suspected bias when they were raters but two suspected bias (political) when they were being rated by their managers • Vast mast majority (15 of 16 respondents) provided unsolicited responses to Question #10 (Other) and indicated that they did not have enough time to implement the PA system • None of the respondents received formal training about the PA systems they were using • 2 received no informal training because they were new hires with considerable experience in conducting PA • Those interviewed were experienced (mean of 16.6 years) • Most inexperienced respondent had 7 years experience Sillup/Porth - 11

12. Must Go beyond Effective PA System to Inspire IBLs • Results reinforce that effective implementation of a PA system is an optimal way to assess an employee’s performance but…………………………………………… • Development must also introduce the employee to aspects of performance that go beyond the objective measures evaluated in results-oriented PA systems if the PA system is to help develop IBLs • These include the performance attributes added to complement the behavior-oriented PA systems being implemented by the companies in this study Sillup/Porth - 12

13. Performance Attributes Added to Behavior-Oriented PA Systems • Examples of this can viewed in three of the four companies participating in this study • Johnson & Johnson • IBM • Aetna • They integrated concerns for employees’ rights into the very fabric of their PA systems and corporate cultures • Despite more recent ethical concerns (e.g., IBM connected to the Holocaust) • Both companies receive strong employee approval for their policies on company attitude surveys Sillup/Porth - 13

14. Allowing Enough Time to Implement PA System is Necessary for Success • Enough time to implement a PA system seems to be a determinant for the success of the system • Furthermore, when time is constrained • Raters revert to familiar objective measures, such as financial metrics, and “tell and sell” approach rather than an information exchange • Raters also become more vulnerable to biases, (e.g., recency effect) • Although most companies require that PA results be discussed with employees, they do not consider PA as an individual objective • Consequently they miss an opportunity to help improve their PA system and to establish the basis for development of IBLs Sillup/Porth - 14

15. Continue: Annual reviews with a mid-year update Feedback has maximum impact when given as close as possible to the action Self-appraisal is very useful Employees who prepare are more satisfied with the PA process and more likely to meet their personal objectives Encourage Participation Managers welcome employees’ ideas Start: Training for Raters Emphasize how to observe behavior more accurately and fairly and not on how to rate or not rate employees’ performance. Set PA as an Objective Objectives that are set emphasize the importance of meeting the objective Implementation Practices - Need to be Continued and Started Sillup/Porth - 15

16. Implications for Further Research • More companies and more raters/ratees from different departments with less experience • Provide the basis for quantitative assessment of these qualitative observations • Larger, more robust dataset would enable other interesting observations • e.g., how managers who “exceed” their annual objectives rate their employees • Consider how PA systems are working globally • Differences between the US and Europe and, especially, Pacific-Rim countries N.B.Contributors to the content of this paper include employees of Aetna, IBM, Johnson & Johnson and Wyeth as well as Susan Givens-Skeaton for her input and introduction to Wayne Casio’s research and Christopher Field for his critical review. Sillup/Porth - 16

17. Connectedness of PA System, Top Management and IBLs • No “correct” PA system to stimulate development of IBLs but one alternative that enables a business to “manage by values” is the Balanced Scorecard Balanced Scorecard PA System for top management Inspired Business Leaders Sillup/Porth - 17

18. Balanced Scorecard Creates Environment for IBLs Financial Measures Owners and Stakeholders’ View Customer Perspective Customers’ View Internal Perspective Employees’ View Balanced Scorecard Future Perspective Continue Adaptation of Source: S. Porth, Strategic Management, A Cross-Functional Approach, 2006 Sillup/Porth - 18

19. Thank You Questions Sillup/Porth - 19

20. Back-Up Slides Sillup/Porth - 20

21. Research Instrument 1.) Does your PA system have clear PS? 2.) How well does your PA system meet the requirements of an effective PA system? • Relevance – identify what makes a difference • Sensitivity – determine effective/ineffective performance • Reliability – platform for consistent judgment • Acceptability – have the support of those using it • Practicality – easy to understand 3.) What type of PA system does you company use? • Behavior-Oriented • Narrative essay • Ranking • Checklist • Critical incidents • Graphic rating scale • Results-Oriented • Management By Objectives • Working Plan and Review Sillup/Porth - 21

22. Research Instrument (continued) 4.) Who rates PA? • Immediate Manager • Peers • Direct Reports • Self-appraisal • Customers • Multi-Rater (360˚-feedback) • Computer Monitoring 5.) Are you aware of any biases in rating? • Leniency and severity • Central tendency • Halo effect • Contrast error • Recency error 6.) Have you had “rater” training? When? 7.) How often is PA conducted? Sillup/Porth - 22

23. Research Instrument (concluded) 8.) Is your company doing some variation of TQM? If so, how does PA fit with TQM? 9.) Have you encountered any barriers to PA? • Lack of top-down support • Lack of employee buy-in • Political barriers • Interpersonal barriers 10.) Are there any other factors that help/hinder implementation of PA? Sillup/Porth - 23