1 / 28

Background: National Benchmark Study TM

Background: National Benchmark Study TM. Much research shows that intrinsic motivation (IM) is a good predictor of an employee’s performance.

virote
Download Presentation

Background: National Benchmark Study TM

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Background: National Benchmark StudyTM Much research shows that intrinsic motivation (IM) is a good predictor of an employee’s performance. For example, the higher the employee’s IM, the higher the cash value of their subsequent sales and the higher the number of loans they issue (Barling, Weber and Kelloway, 1996), the greater the likelihood of sensible risk-taking, goodinnovation (Ganesan and Weitz, 1996; Sathe, 1989) and even a more enduring impact from learning interventions (Geller, Rudd, Kalsher and Streff, 1987), outcomes that are seen in many – but admittedly not all cases, -- regardless of the level of the employee’s pay (Bassett-Jones and Lloyd, 2005). As expected, a number of meta-analyses have also found good outcomes stemming from high IM and the HR practices that foster it; for example, these studies typically associate IM with higher levels of subsequent staff retention, higher profit, and fewer on-the-job injuries (Harter, Schmidt and Hayes, 2002; Wright, Gardner, Moynihan and Allen, 2005). Moreover, similar research has also found a link between high IM and outcomes such as a better ability to put in additional discretionary effort for the work team, mentor new staff members, and avoid waste—outcomes that typically have little impact on the individual employee’s personal performance metrics (Saks, 2006). It makes sense then, that effects from IM should be manifest in the performance metrics of the whole organization, even using a metric as diffuse and multifaceted as our focus: The Cash Value of Dividends Paid to Stockholders at the End of the Fiscal Year.

  2. Method Assessment Tool: An anonymous survey about working conditions comprised of 11 rating questions and 2 demographics. Rating questions followed the following format: “In your opinion, throughout your company during the last year, what percent of the teams had great communication?” Responses ranged from 0% 10 100% in 10% steps. Questions pertained to Teamwork, Leadership, Training, Pay & Benefits, and Ethics & Fairness. (5 cross validation questions were also included; they asked about perceived efficiency, perceived quality, perceived value and/or profit, perceived customer satisfaction, and perceived employee motivation.) Demographics asked about job level (3 options) and company name. Sample Frame: 1000 companies are contacted each year of the study and asked to participate; companies were chosen by the Wall Street Journal for their WSJ1000 analysis, which looks at stock return during the last 1, 3, 5,and 10 years. The WSJ100 covers 22 million employees at these publicly held companies, and accounts for 98% of the nation’s GDP. Time Period for the Study: We first surveyed employees in January of 2001; we also surveyed in 2002, 2003, 2004, 2005, and 2006. During the 7 years of the study 3490 employees from 841 corporations took the NBS survey. Format: During the first two years of the study we worked with the University of Michigan to administer the survey by phone; during the remaining years we administered the survey using paper and a web interface; respondents could chose to use either a paper survey or the web. Sampling: When the survey was administered by phone we used a simple random sample to select 5 employees from a list of 100 phone numbers generated by altering the final two digits of the corporation’s main phone number, stopping if we completed 5 surveys or were told that company policy prohibited outside polls or encountered three consecutive refusals. When the survey was administered by web and paper the HR director at participating firms created an alphabetical list of all employees, and selected the first employee from each letter in the alphabet, continuing to loop through the list until 100 employees were selected for the survey.

  3. RESULTS: Reliability & Validity

  4. Evidence of Validity: The Survey Generates Normal Curves 55 60 65 70 75 80 85 90 95 100 The curve above shows Perceived Quality, averaged by Industry; many distributions in the data are similar. Reliability coefficient is very high (Cronbach’s Alpha = .88)

  5. Evidence of Validity: Perceived Employee Motivation Tracks Actual EMI 77.5 75 72.5 EMPLOYEE MOTIVATION INDEX (EMI) 70 67.5 55 60 65 70 75 80 PERCEIVED EMPLOYEE MOTIVATION Statistical Note: Plot shows means by Sector; n = 13, p < .00001, r = .89. Data come from NBS interviews conducted in 2004.

  6. Evidence of Validity: Perceived Customer Sat Correlates with Actual Customer Sat The scores for Perceived Customer Satisfaction from the NBS survey are highly correlated with scores for Actual Customer Satisfaction from the American Customer Satisfaction Index (ACSI). The finding is significant because the two surveys were administered to different groups of respondents (employees and customers respectively) over the course of several years using surveys that were entirely different in wording and methodology…a primary requirement for convergent validity. 85 80 75 70 American Customer Satisfaction Index 65 60 55 30 40 50 60 70 80 90 100 NBS Q11: Perceived Customer Satisfaction Statistical Note: Correlation is shown between NBS Question 11 and ASCI Scores aggregated by participating companies from 2001 to 2005; n = 17, 4250 (companies, ACSI interviews); r = .31, p < .0001.

  7. Evidence of Test-Retest Reliability: We find no significant difference between formats 80 70 60 50 EMI (Q1-Q6) 40 30 20 All Pairs PHONE WEB Tukey-Kramer 0.05 In a test-retest comparison of data from respondents who were willing to have their responses verified (Response Rate = 30%. i.e., 22 out of 74) we found no significant difference between mean EMI scores from our web-based interface and our phone interviews. Statistical Note: Surveys and retests by phone were completed during 12/2003 and 1/2004; R-Square = 0; n = 44, F = .001; Mean EMI using Web = 66.8; Mean EMI using phone = 66.9; Least Significant Number is 164,251 at alpha = .05.

  8. RESULTS: Preliminary Evidence of Linkages

  9. In 2001 EMI Predicted 1-year Stock Return for the Remainder of the Fiscal Year 90 70 50 30 10 One-Year Return % -10 -30 -50 -70 -90 40 50 60 70 80 90 100 Mean EMI (Employee Motivation Index) Statistical Note: Calls were made from 1/6/2001 to 6/30/2001; Stock data come from the WSJ’s Shareholder Scoreboard 2/25/02. 11

  10. In 2002 EMI Predicted the NEXT YEAR’S Return to Stockholders 250 200 150 100 Stock Return Next Year (%) 50 0 -50 10 20 30 40 50 60 70 80 90 100 Mean EMI (Leverage) The higher the Employee Motivation Index, the higher the total Stock dividend paid to shareholders during the NEXT fiscal year. Specifically, every 5 point rise in EMI yielded an additional 2% return the next year. Statistical Note: Calls were made during the Winter of 2002; Stock data come from the WSJ’s Shareholder Scoreboard published on 3/8/04 for FY 2003. R2 = .49; F = 10.2; aggregated by company, n = 272, 796 controlling for Industry & Headcount; p < .0001; Beta = .004 p < .003.

  11. Tukey-Kramer 0.05 In 2003 EMI AGAINPredicted the NEXT YEAR’S Return to Stockholders 70 60 50 40 2003 EMI (Employee Motivation Index) 30 20 All Pairs LOST GAINED I YEAR STOCK RETURN FY2004 (LOST or GAINED) Companies whose stock LOST value during FY 2004 had lower scores on the Employee Motivation Index (EMI) in the previous year than companies whose stock Gained value. For the 4th consecutive year, BenchmarksTM for the company’s working conditions have predicted the company’s financial performance during THE NEXT YEAR. Statistical Note: Surveys were filled out during the Winter of 2003; Stock data come from the WSJ’s Shareholder Scoreboard published on 2/28/05 for FY 2004. R2 = .45; p < .0001; aggregated by company, n = 10, 210. Correlation is also very high and significant: r = .70. The improvement in Ethics & Fairness bodes well for next year’s Sales (r = .72, n = 10, 210) IACS (r = 24, n = 10, 210) and EBITDA (r = 25, n = 10, 210.) all p’s < .00001.

  12. In 2004 EMI also Predicted the NEXT YEAR’S Return to Stockholders 0.4 0.3 Leverage Residuals 0.2 1-Year Return FY04 0.1 0.0 50 60 70 80 90 Mean (EMI Rating 2003) Leverage, P<.0001 Statistical Note: Surveys were filled out from 10/2004 to 2/2005; Stock data come from the WSJ’s Shareholder Scoreboard published on 2/28/05 for FY 2004. R2 = .50; F = 154.1; p < .0001; aggregated by company, n = 7, 313. Beta = 0.008, p < .0001, controlling for Headcount.

  13. In 2005 EMI again Predicted NEXT YEAR’S Return to Stockholders (1 of 2) 0.50 0.25 0.00 Leverage Residuals 1 - Year Return NBS 2006 (FY 05) -0.25 -0.50 45 50 55 60 65 70 75 Mean(EMI) Rating 2004 Leverage, P<.0001 The higher the EMI, the higher the Stock dividend during the next fiscal year, although in this case, survey data collection started later (Oct 2005) and continued later (Mar 2006) than planned; for example, ratings during this cycle reflected perceived working conditions as early as the year from Oct ‘04 to Oct ’05 and as late as the year from Mar ‘06 to Mar ‘06. Accordingly, EMI predicted Stock Return during FY 2005 for no more than 3 months, and overlapped with it for the remaining 9 months. However… Statistical Note: Surveys were filled out from 10/2005 to 3/2006; Stock data come from the WSJ’s Shareholder Scoreboard published on 2/27/06 for FY 2005. R2 = .99; F = 88509; p < .0001; aggregated by company, n = 9, 231, controlling for Headcount and Sector as determined by Yahoo.

  14. In 2005 EMI even Predicts the following year’s Stock Return as well (2 of 2) 0.4 0.2 0.0 1 - Year Return (FY 06) Leverage Residuals -0.2 -0.4 0 25 50 75 100 Mean(EMI) Rating 2004 Leverage, P<.0001 …Results are substantively similar if we run the analysis using financial data from 2006. In this case, EMI from 2004 and 2005 is examined to see if it predicts stock return during the 2006 fiscal year. Results are virtually unchanged: The higher the Employee Motivation Index, the higher the subsequent Stock dividend, even though some of the EMI ratings were separated from the fiscal data by a 3-month hiatus, and EMI ratings never overlapped with stock return for more than 3 months. Statistical Note: Surveys were filled out from 10/2005 to 3/2006; Stock data come from the WSJ’s Shareholder Scoreboard published on 2/26/07 for FY 2006. R2 = .36; F = 65.4; p < .0001; aggregated by company, n = 9, 231. Beta = 0.02, p < .0001, controlling for Headcount.

  15. 17

  16. Appendix Additional Linkages Between Aspects of IM & Financial Metrics

  17. Training Drives Earnings Before Income Taxes Depreciation & Amortization (EBITDA) 1250 1000 750 500 FY05 Leverage Residuals EBITDA Total ($M OK) 250 0 30 40 50 60 70 80 Mean(Q4 FY04) TRAINING Leverage, P<.0001 The higher the Training score from the NBS survey in 2004, the better EBITDA in 2005. Statistical Note: Surveys completed during 2004; financial data summarized at the end of FY 2005, and come from SEC reports, Yahoo, and similar sources; stock return data from Hoovers. R-Square = .97; aggregated by company, n = 6, 312 controlling for Headcount; p < .0001; Beta = 4.11

  18. Leadership Drives Profit Margin 0.30 0.25 0.20 0.15 Profit Margin % FY05 Leverage Residuals 0.10 0.05 0.00 50 55 60 65 70 75 80 85 Mean(Q2 FY04) LEADERSHIP Leverage, P<.0001 The higher the Leadership score from the NBS survey in 2004, the better the Profit Margin in 2005. Statistical Note: Surveys were completed during 2004; financial data were summarized at the end of FY 2005, and come from SEC reports, Yahoo, and similar sources; stock return data come from Hoovers. R-Square = .82; aggregated by company, n = 6, 312 controlling for Headcount; p < .0001; Beta = 4.11

  19. Communication Drives Income Available for the Common Stock (IACS) 600 500 400 300 FY05 Leverage Residuals IACS Total ($M OK) 200 100 0 50 55 60 65 70 75 80 Mean(Q3 FY04) COMMUNICATION Leverage, P<.0001 The higher the Communication score from the NBS survey in 2004, the better IACS in 2005. Statistical Note: Surveys were completed during 2004; financial data were summarized at the end of FY 2005, and come from SEC reports, Yahoo, and similar sources; stock return data come from Hoovers. R-Square = .96; aggregated by company, n = 6, 312 controlling for Headcount; p < .0001; Beta = 12.6

  20. Short-Term Stock Return Drives Subsequent Ratings of Pay & Benefits 0.6 0.5 0.4 0.4 0.3 0.2 0.2 0.1 (FY05) Leverage Residuals (FY05) Leverage Residuals 1 - Year Return HOOVERS 3 - Year Return HOOVERS 0.0 0.0 -0.1 -0.2 -0.2 30 40 50 60 70 80 90 30 40 50 60 70 80 90 Mean(Q5 FY04) PAY & Mean(Q5 FY04) PAY & BENEFITS Leverage, P<.0001 BENEFITS Leverage, P<.0001 The better the short-term Return to Shareholders in FY 2005 (averaged over the previous one and three years), the higher the ratings for perceived Pay & Benefits from employees taking the NBS questionnaire. The finding suggests that, in general, better financial performance leads to better pay & benefits. Statistical Note: Surveys were completed during 2004; financial data were summarized at the end of FY 2005, and come from SEC reports, Yahoo, and similar sources; stock return data come from Hoovers. R-Square = .81; aggregated by company, n = 6, 312 controlling for Headcount; p < .0001; Beta = .03 for both 1-Year Return and 3-Year Return.

  21. Long-Term Stock Return and ROI Predict Subsequent EE Motivation 0.5 0.4 0.20 0.3 0.15 0.2 0.1 0.10 Leverage Residuals 5 - Year Return HOOVERS (FY05) ROA % (FY05) Leverage Residuals 0.0 0.05 -0.1 0.00 -0.2 50 55 60 65 70 75 50 55 60 65 70 75 Mean EMI (FY 04) Leverage, P<.0001 Mean EMI (FY04) Leverage, P<.0001 Good Long-term Stock Return (averaged over 5 years) predicts subsequent EMI. The result suggests that good corporate governance and a sound business model lead to good stock return and high employee motivation. The finding is especially important because many of our analyses also show that high employee motivation contributes to high subsequentReturn on Assets. Statistical Note: Surveys were completed during 2004; financial data were summarized at the end of FY 2005; stock return data come from Hoovers. R-Square = .96; aggregated by company, n = 6, 312 controlling for Headcount; p < .0001; Beta = .001 for subsequent ROA; Beta = .86 for concurrent CSAT (not shown); Beta = .85 for concurrent Quality (not shown); Beta = .02 for 5-Year Stock Return. P value for all Betas = .0001.

  22. The Linkage Seems to be Bidirectional: Growth in Stock Return Predicts Growth in Subsequent EMI 100 DELTA in 1 yr return (fy 2001 minus fy 2000) Leverage Residuals 0 -40 -30 -20 -10 0 10 20 30 40 DELTA EMI (EMI of 2002 minus EMI of 2001) Leverage, P=0.0137 24 From NBS RESULTS 09 summarizing Data from 2002 using calls of 2001

  23. References & Additional Citations Heskett et al, (1994) Putting the Service-Profit Chain to Work, HBR, March-April Morrel-Samuels (2002) Getting the Truth into Workplace Surveys, HBR, February Vol. 80 No 2 pp 111-118. Kaplan & Norton (1992) The Balanced Scorecard, HBR, January-February Morrel-Samuels (2003) Web Surveys’ Hidden Hazards, HBR, July Vol. 81 No 7 pp 16 - 17. Kaplan & Norton (1996) The Balanced Scorecard: Translating strategy into action, Boston: Harvard Business School Press Cohen & Cohen (1983) Applied Multiple Regression/Correlation... Hillsdale: Erlbaum Cook & Campbell (1979) Quasi-Experimentation: Design & analysis issues for field settings, Chicago: Rand McNally Bryk & Raudenbush (1992) Hierarchical Linear Models: Applications and data analysis methods. [Vol. 1: Advanced Quantitative Techniques in the Social Sciences.] Newbury Park, CA: Sage.

  24. References & Additional Citations (continued) Barling, J., Weber, T., & Kelloway, E. K. (1996). Effects of transformational leadership training on attitudinal and financial outcomes: A field experiment. Journal of Applied Psychology, 81(6), 827-32. Morrel-Samuels, Palmer & Jacobson, Peter D., "Using Statistical Evidence to Prove Causality to Non-Statisticians" (July 5, 2007). Available at SSRN: http://ssrn.com/abstract=995841 Saks, A. M. (2006). Antecedents and consequences of employee engagement. Journal of Managerial Psychology, 21(7), 600-619. Cook, Colleen, Heath, Fred, & Thompson, Russel L. (2000). A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys. Educational and Psychological Measurement, Vol. 60, No. 6, 821-836 Richman, Wendy L, & Quiñones, Miguel A. (1996) The Effect of Task Engagement and Experience. Journal of Applied Psychology. Vol. 81, No. 5, 512-524 Webster, Jane, & Compeau, Deborah. (1996). Computer-assisted versus paper-and-pencil administration of questionnaires. Behavior Research Methods, Instruments & Computers. 28 (4), 567-576.

  25. References & Additional Citations (continued) Sathe, V. (1989). Fostering entrepreneurship in the large, diversified firm. Organizational Dynamics, 18(1), 20-32. Bassett-Jones, N., & Lloyd, G. C. (2005). Does Herzberg's motivation theory have staying power? Journal of Management Development, 24(10), 929-943. Ganesan, S., & Weitz, B. A. (1996). The impact of staffing policies on retail buyer job attitudes and behaviors. Journal of Retailing, 72(1), 31-56. Wright, P. M., Gardner, T. M., Moynihan, L. M., & Allen, M. R. (2005). The relationship between HR practices and firm performance: Examining causal order. Personnel Psychology, 58(2), 409-446. Harter, J. K., Schmidt, F. L., & Hayes, T. L. (2002). Business-unit-level relationship between employee satisfaction, employee engagement, and business outcomes: A meta-analysis. Journal of Applied Psychology, 87(2), 268-279. Geller, E. S., Rudd, J. R., Kalsher, M. J., & Streff, F. M. (1987). Employer-based programs to motivate safety belt use: A review of short-term and long-term effects. J. of Safety Research, 18(1), 1-17.

  26. Contact Information If you have any questions about the analysis or design of the National Benchmark SurveyTM please call Dr. Palmer Morrel-Samuels, President of Employee Motivation & Performance Assessment, Inc. at 734-368-3348 between 9am and 6pm EST. Questions, comments, or suggestions are always welcome. www.WorkplaceResearchFoundation.org palmer@umich.edu

More Related