The Situational Interview

1. The Situational Interview www.mubeena.net www.mubeena.net

2. www.mubeena.net Job-related situational questions Situations are specific on-the-job activities Applicants put in hypothetical situations Questions can refer to past experience or future intentions Differs from structured format by benchmarking applicant responses Definition

3. www.mubeena.net EvolutionTree Redefinition of Selection Tool “A valid test cannot be developed until the organization agrees upon an acceptable definition (measure) of employee behavior.” Latham & Wexley, 1982 Workforce Prediction Quality of workforce will deteriorate Dyer, 1981 Legal Issues Selection Process Dilemma Griggs vs. Duke Power, 1971

4. www.mubeena.net Latham, Saari, Pursell, & Campion, 1980 Examined reliability and validity of situational interview by conducting three studies Study 1: entry-level position, Study 2: first-line supervisory position, Study 3: concurrent predictive validity Results for Study 1: Interview scores were significantly correlated with every performance criterion including the overall global rating Results for Study 2: Interview scores correlated significantly with three of four criteria – safety, work habits, org commitment Results for Study 3: Interview scores significantly correlated with composite job performance scores, even for women and black employees Research Literature

5. www.mubeena.net Conclusions of Latham et al. (1980) Intentions correlate with behavior Comprehensive job analysis reflect content validity Interviewee motivation influenced by face validity Inter-rater reliability high because interviewers themselves developed scoring key Emphasis on critical behaviors rather than traits Past behavior based items could lessen dishonest responses, it can be verified by previous employer. However, adverse impact must be kept in check Research Literature

6. www.mubeena.net Pulakos & Schmitt, 1995 Compared validities of experience-based (past performance) questions and situational (future intentions) questions Results: Past performance is better predictor of job performance Lack of Validity for Situational Items Attributed to: Responses were evaluated at end of interview. It is necessary to make the ratings immediately Prior studies indicate that situational interviews involve lower-level jobs. In this study, the job was complex and demanding Major conclusion was that interviews result in lower levels of adverse impact because they measure cognitive as well as non-cognitive performance dimensions Research Literature

7. www.mubeena.net Maurer & Fay, 1988 Two hypotheses: Rater training Inter-rater agreement Situational Inter-rater agreement Interpretation of results: Situational interviews are more effective in producing higher inter-rater agreement, even with little or no training, because of specific rating scales There was so significant main effect of training Situational interview is more cost-effective strategy in comparison with conventional structured interview Research Literature

8. www.mubeena.net Was first developed and used by John Flanagan and his students at the University of Pittsburgh in the late 1940s and early 1950s Used to identify job behaviors that differentiate successful performance from unsuccessful performance Can be useful in job analysis as well as training and performance appraisal The development of the graduate situational interview was based on this technique Critical Incident Technique

9. www.mubeena.net Critical incidents were identified by reviewing the job analysis (CMQ) Behaviors that could exemplify either good or poor behaviors of graduate students were chosen Fourteen job dimensions were identified: Prioritizing Leadership Stress management Ethics Ability to work with others Public speaking Receiving feedback Altruism Self Assessment Diversity Decision making Retention Group involvement Participation Critical Incident Development

10. www.mubeena.net The critical incidents were used to form the questions to be asked in the situational interview The beginning of each question stem entailed a description of a circumstance involving a critical incident Example: Critical Incident – Decision Making Example: You are placed at a company in which your supervisor is critical and provides no direction to complete a task The questions ended with a general proposition Example: “How would you handle this situation?” Seventeen questions were written using this format Each critical incident that we identified had a corresponding question Two of the critical incidents had more that one corresponding question, namely stress management and ethics Development of Situational Interview Questions

11. www.mubeena.net To facilitate the scoring process a behaviorally anchored rating scale was developed for each question The behavioral scales were developed by brainstorming and determining KSA relevance and importance as rated by SMEs (job incumbents) Our anticipation of the responses that we would receive was used to develop a five point scale The five point scales included examples of high, average and low responses The examples that we agreed represented high, average and low responses were used as behaviors on the scale BARS Development

12. www.mubeena.net 5: Applicant tries to talk to the boss 4: Applicant tries to set own goals and direction no resolve of conflict. 3: Applicant says nothing. 2: Applicant complains, does not start the job until the issue is addressed. 1: Applicant leaves the job/talks to boss in a negative fashion. Example

13. www.mubeena.net Our questions and behavioral scales were presented to a focus group The focus group (subject matter experts) recommended the following: - add multitasks to the scales - reword ambiguous and vague questions - make sure that each question corresponds to the critical incident it is describing in a situation Focus Groups

14. www.mubeena.net Each response on the behavioral scale had a corresponding score Scores were rated from one to five - a 1 or 2 represented a low response - a 3 represented an average response - a 4 or 5 represented a high response The interviewers choose the response that best represents the interviewees answer A total score for the interview can be obtained by summing the ratings for each question Scoring of the Situational Interview

15. www.mubeena.net The Ebel method was used to determine the cutoff score - rate each item (1-5) - determine the percentage of items a minimally qualified candidate would respond to correctly (a correct response to each item represented a score of 3 or more) - multiply this percentage by the number of items (3 x 17 = 51) - the cutoff score becomes 51 for the situational interview Development of Cutoff Score

16. www.mubeena.net Participants: 30 I/O graduate students Two groups of raters: 3 and 2 Selection Tool was administered twice over a 2 week period. The Tool was administered in a classroom setting with other selection tests being administered at the same time. Administration

17. www.mubeena.net Random Error: Noise/Disruption Effects Exposure to test questions before administration Fakeablity Rater effects(Halo, Contrasts, Leniency Effects) Possible Threats To Validity

18. www.mubeena.net Mean=64.21 Standard Deviation=5.78 Range=27.67 Skewness=-.845 Descriptive Data

19. www.mubeena.net Score Distribution

20. www.mubeena.net Correlation with -Graduate GPA r=.284, p=.064 -Psychology 249 r=.360, p=.025 -Psychology 283A r=.381, p=.019 -Undergraduate GPA r=.252, p=.089 -GRE r= -.007, p=.485 Validity

21. www.mubeena.net Standardize scores Reliability and more validity analysis Analyze individual questions Near Future

22. www.mubeena.net How can we modify the interview to make it a better instrument for selecting graduate students? separate information gathering and evaluation, evaluate after gathering modify behavioral anchors re-categorize questions so that questions which co-vary are in the same category Distant Future