WorkKeys Innovations: A Holistic Solution for WIRED West Michigan

1. WorkKeys Innovations: A Holistic Solution for WIRED West Michigan Steve Robbins, AVP, Applied Research, ACT, Inc.

2. Overview • Why we should care about combining cognitive- and personality-based measures • National Career Readiness Certificate “Plus” • What we know from workforce and educational literatures • Differential test strategies along the entire continuum of employment

3. Why Now? • Market Need: SHRM National Study • Body of research informs how to optimize Personal Skill Assessments • ACT strength in cognitive & non-cognitive assessment (John Holland, VP Research in 1960’s) • Solution-focused approach

4. Businesses want integration of Cognitive & Personality Constructs SHRM: Applied Skills and Basic Knowledge: Combining and Ranking For new entrants with a two-year college/technical school diploma, applied skills are four of the top five “very important” skills in combined ranking with basic knowledge and skills. Casner-Lotto, J. & Barrington, L. (2006)

5. Solutions are Need Driven: Continuum of Employment

6. WorkKeys Assessment Solutions: Pyramid for Success • Ensure work and training readiness (WorkKeys Foundational Skills) • Provide employers another source of information for selection (“Performance”) • Give a “snap shot” of strengths and areas of improvement across key response tendencies or domains (“Talent”) • Promote career exploration of job fit (“Fit”)

7. Pyramid for Success • FIT • Match individual interests/values to work environment • Enhance job persistence & satisfaction • Develop Talent pool to meet needs • TALENT • Benchmarking for selection • Coaching & Development • Compound Indices: Sales, Managerial, Leadership, Safety • PERFORMANCE • General Work Performance: Productivity, Absenteeism, Complaints about conduct • Safety and Risk Reduction • FOUNDATIONAL SKILLS • Job Analysis – identifies the skills and skill levels needed to be successful on the job • Assessments – show the current skill levels of an individual • Training – helps individuals and employers correct skill gaps

8. 6 5 5 Skill Gap Benchmark Levels Required 4 4 4 3 3 3 Worker’s Skill Level 2 1 Applied Math Reading Locating Information National Career Readiness Certificate Currently, the WorkKeys system assesses foundational skills, such as reading and locating information, using a three-step process of job profiling, assessment, and skill-gap training.

9. National Career Readiness Certificate “Plus” • A flexible solution to meet state and system needs • Georgia example • Talent for coaching & development • “Fit” for career exploration • Alterable variables allow for intervention & guidance

10. Cognitive Ability Tests“Rule” • General vs. specific test effect sizes • Adverse impact issues can be ameliorated with specific tests: • Task analysis • Job-specific tests • Combinational use of cognitive and non-cognitive tests * Brown, Le, & Schmidt (2006) **Salgado et al. (2003)

11. Personality Tests“Add Value” • Validity Estimates: 1 From Schmidt et al. (2007) using indirect range restriction

12. What are We Talking About? Personality Test Constructs • The Big Five: Emotional Stability, Extraversion, Agreeableness, Conscientiousness, Openness to Experience • Specific or Facet-level Traits: Carefulness, Discipline, Influence, Order, Sociability, Drive, Creativity • Compound Traits: Service Orientation, Integrity, Managerial Potential, Teamwork

13. Personality Test Issues • Selection vs. Coaching and Development as Differential Applications • Selection Approach • Recommend multiple hurdles and/or top-down approach • Adverse Impact Limited • Differential Outcomes (Rotundo & Sackett, 2002) • Task Performance (Technical Core) • Organizational Citizenship Behavior (Contextual Performance) • Counterproductive Work Behavior • Add Satisfaction/Tenure as another Key Outcome

14. Combining Personality & Cognitive Ability Tests • Level of correlations are low: GMA x C = .02 GMA x ES = .17 Math x C = -.15 Math x ES = .17 Reading x C = -.05 Reading x ES = .11 (Ackerman & Heggestad, 1997)

15. Combining Personality & Cognitive Ability Tests Creating opportunity for incremental validity especially as criteria vary Correlations between general cognitive ability and personality tests and measures of job performance in Project A Cog Pers Both Criteria .63 .26 .67 Core technical proficiency .65 .25 .70 General Soldiering Proficiency .31 .33 .44 Effort and Leadership .16 .32 .37 Personal Discipline .20 .37 .42 Physical fitness and military bearing (McHenry, Hough, Toquam, Hanson, & Ashworth,1990)

16. Educational Examples 1 Correlation of fitted probabilities and dichotomous retention outcome 2 SRI scale scores and ACT Composite score used as predictors

17. Educational Examples 1 Multiple R from linear regression model 2 SRI scale scores and ACT Composite score used as predictors

18. ACT: Foundational and Personal Skills Assessments measure Different Applied Job Skills:

19. “If they staged a slowdown, how would we know?” Harvard Business Review. March 2007. p. 90

20. Performance Score Report

21. “It was about here, wasn’t it, Ed, when you came on board as sales manager?” Harvard Business Review. March 2007. p. 90

22. Talent Score Report

24. FIT Score Report

26. Selection Solutions • Reducing Risk • Task Competence through WorkKeys and job profiling • General Work & Safety • Increasing Tenure • Task Competence through WorkKeys and job profiling • Fit • Getting the Right Person • Talent Benchmarking • Past Work Performance Record

27. Coaching & Development Solutions • Career Exploration • Task Competency • Fit • Leadership Development • Talent • Fit • Teamwork • Talent

28. Return on Investment Approximations under Various Scenarios Notes: Selection % = the percentage of the candidate pool selected for hire, Candidate Success % = the percentage of the candidate pool that would be successful if hired, Selected Success % = the percentage of the selected candidate pool that will be successful, Cost per failure = the average cost for each unsuccessful employee relative to each successful employee, ROI per 100 candidates = the average return on investment for the selection procedure assuming a $15 fee per candidate.

29. Final Thoughts • Adverse Impact may be reduced when combining tests • Still recommend “multiple hurdles” approach • Incremental Validity Research underway: • Combination of Task & Personality measures • Differential work outcomes • ROI x Solution

30. References • Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121, 219-245. • Brown, K. G., Le, H., & Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14, 87-100. • Casner-Lotto, J. & Barrington, L. (2006). Are they really ready to Work? Society for Human Resource Management. http://www.shrm.org/hrresources/surveys_published • McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology, 43, 335-354. • Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598-616. • Rotundo, M., & Sackett, P. R. (2002). The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach. Journal of Applied Psychology, 87, 66-80. • Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573-605. • Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. • Schmidt, F. L., Shaffer, J., & Oh. I. (2007). Reassessing the Relative Importance of Cognitive Ability and Personality in Job Performance and Training Performance: Some Surprising New Research Findings. Paper presented at the 2007 ATP conference, Palm Springs, CA. Feb. 6.

31. Incorporating Foundational and Soft Skill Assessments For questions regarding this presentation or for further information contact: Steve Robbins at 319-337-1227 or steve.robbins@act.org, Gary Nolan at 319-337-1526 or gary.nolan@act.org,