The Development of a Delayed Retrospective Report Method to Increase the Reliability and Validity of Verbal Reports about Past Events. David W. Eccles, Paul Ward, Kevin R. Harris, Lauren Tashman, K. Anders Ericsson, and Laura B. Hassler Expert Performance Research Team,
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
David W. Eccles, Paul Ward, Kevin R. Harris, Lauren Tashman,
K. Anders Ericsson, and Laura B. Hassler
Expert Performance Research Team,
Human Performance Laboratory
Learning Systems Institute,
The Florida State University, USA
Reports might look like “There’s the P, then there’s err Q, then R, S, and then T, it’s T”.
Experimenter: What was the first thought you remember having while coming up with the answer, and the next, and the next, and so on?
Participant: “I saw a P, and then there was a Q, R, S, and finally a T, it was T”.
- only report those thoughts and actions that can be recalled with certainty
- avoid interpreting, explaining, or summarizing thoughts or actions
Step 1: Initial overall description of event obtained and establishment of sub-events and their timeline
Section of transcript:
July 11 2005
Expert Police Officer “Truman”
12 expert (SWAT, TAC) and 12 novice (cadet) police officers (but I’m not going to report here on expert-novice differences)
Initial training in think aloud and IRR techniques with established exercises
In a simulator, officers equipped with modified gun, exposed to 20 film clips depicting representative and stressful police scenarios, requiring shoot/no shoot decision
Video captures behavior of officer – handles gun, unholsters gun, points and aims gun relative to events occurring on film – all verbal and non-verbal commands
For each shot, computer captures initial trigger squeeze time point, full trigger depression time point, and gun shot placement relative to events occurring on screen
IRR captures thoughts experienced during scenario
Video of whole body behavior
Video record of trigger behavior
DRR is then performed about thoughts and actions undertaken during two
(a priori determined) stressful scenarios, indicated with*