High Fidelity Patient Simulation: Process and Outcomes to Improve Competency Education and Validation

1. High Fidelity Patient Simulation:Process and Outcomes to Improve Competency Education and Validation Susan Sportsman, RN, Ph.D., Dean Midwestern State University College of Health Sciences and Human Services Wichita Falls, Texas, USA

2. Texas Higher Education Coordinating Board (THECB) Grant Purpose of 2004 funding To provide funding to eligible institutions to address the shortage of RNs by developing innovative educational projects for initial RN-licensure nursing students and faculty Primary Outcome: Increase enrollment

3. North Central Texas Health Care Consortium Midwestern State University (MSU)-BSN Vernon College (VC)-ADN United Regional Health Care System (URHCS) (350 bed regional hospital)

5. Purposes of $1.27 Million Grant Pilot the implementation of a collaborative regional simulation center Increase the number of students admitted to clinical courses at MSU and VC by 56. Evaluate the effectiveness of such a center upon graduates� perceived clinical competence. Determine the cost-effectiveness of such a center.

6. Structure of Regional Simulation Center (RSC) 3,410 square feet renovated nursing unit at URHCS 7 high fidelity patient simulators (4 adults and 3 infants/child manikins) 4 clinically strong BSN lab mentors, supervised by a MSN director Responsible for providing competency education and validation to BSN, ADN, and hospital clinicians.

7. Strategies for Development and Evaluation of Scenario Based Simulations Marcy S. Beck, RN, MSN Co-director Regional Simulation Center Midwestern State University

8. Literature Supports Using Simulation as a Teaching Strategy Active engagement Development of psychomotor skills and critical thinking skills Interactive Create a real and risk free environment Explore feelings and outcomes Communication and teamwork skills Provides for knowledge application Increase in learner satisfaction and confidence

9. Getting Started� Large investment of faculty time Write simulation programs Limited number of students at one time Faculty inexperience with high fidelity simulators What do faculty need? Support from Simulation Center staff Educational programs Writing scenarios Operating the simulator Nursing faculty shortages, limited clinical settings available to students, nursing shortages, and higher acuity levels in our patients increases the need for simulation. So why is it so hard to get started? There are many schools and institutions with high fidelity simulators in the closet, unused. Even though Deans of schools and administrators provide the budgetary support, they are not used. Faculty attitudes for using simulation are generally positive. They see that simulation provides a realistic patient care experience which promotes critical thinking in a safe environment. So again why is it difficult to get started? Large investment of faculty time Write simulation programs Evidence based Correspond to their textbook or information presented in the classroom Review of scenario at least annually for up to date information or maybe changed the textbook Faculty inexperience with high fidelity simulators How do they work? What can they do? A new language to learn What do faculty need? Support from simulation center staff Operations Scheduling Educational programs to address the faculty gap in education Nursing faculty shortages, limited clinical settings available to students, nursing shortages, and higher acuity levels in our patients increases the need for simulation. So why is it so hard to get started? There are many schools and institutions with high fidelity simulators in the closet, unused. Even though Deans of schools and administrators provide the budgetary support, they are not used. Faculty attitudes for using simulation are generally positive. They see that simulation provides a realistic patient care experience which promotes critical thinking in a safe environment. So again why is it difficult to get started? Large investment of faculty time Write simulation programs Evidence based Correspond to their textbook or information presented in the classroom Review of scenario at least annually for up to date information or maybe changed the textbook Faculty inexperience with high fidelity simulators How do they work? What can they do? A new language to learn What do faculty need? Support from simulation center staff Operations Scheduling Educational programs to address the faculty gap in education

10. What Do We Do? Learning objectives Patient initially admitted in diabetic ketoacidosis. She is better and has been transferred from the ICU to the medical-surgical unit. Patient becomes hypoglycemic and requires recognition and treatment by the learner Later, the patient becomes hyperglycemic and requires recognition and treatment by the learner Debriefing points What do we do? We have a unique working relationship with the members of the consortium. The Simulation staff facilitate the scenarios with the students in our center. We provide feedback on content of scenarios (what works and what does not) and feedback on their students performance. Faculty from the educational institutions and staff from the hospitals work together with the Simulation staff to develop the scenarios. Although we are nurses, we view faculty as the experts for their students. They know what objectives are needed for the curriculum. We are the experts in simulation. We know how to create realism, scenarios, and what the simulators can or cannot do. The faculty provide us with the learning objectives, general topic and progression that they would like the scenario to go in and the debriefing points. We take it from there. We write the scenarios. Once they are approved by faculty, we run the scenarios for their students in our center. We have prebuilt templates with all the required information. Patient status, progression, teaching points to stress, cues for the learners, and debriefing questions or points. This way all Simulation staff have the same script to follow. Topics for scenarios are chosen by the faculty. They may relate to content being presented in the classroom, recommendations from licensing boards, high miss questions on exams, concepts that students have difficulty with historically, trends identified through quality programs, and hi risk/low volume incidences. Once a topic is chosen and objectives are written it is time to getting writing. The faculty will provide us with the basic information and we take it from there. It is easier to write a scenario if you have access to medical records. This allows you to not come up with lab values and vital signs.What do we do? We have a unique working relationship with the members of the consortium. The Simulation staff facilitate the scenarios with the students in our center. We provide feedback on content of scenarios (what works and what does not) and feedback on their students performance. Faculty from the educational institutions and staff from the hospitals work together with the Simulation staff to develop the scenarios. Although we are nurses, we view faculty as the experts for their students. They know what objectives are needed for the curriculum. We are the experts in simulation. We know how to create realism, scenarios, and what the simulators can or cannot do. The faculty provide us with the learning objectives, general topic and progression that they would like the scenario to go in and the debriefing points. We take it from there. We write the scenarios. Once they are approved by faculty, we run the scenarios for their students in our center. We have prebuilt templates with all the required information. Patient status, progression, teaching points to stress, cues for the learners, and debriefing questions or points. This way all Simulation staff have the same script to follow. Topics for scenarios are chosen by the faculty. They may relate to content being presented in the classroom, recommendations from licensing boards, high miss questions on exams, concepts that students have difficulty with historically, trends identified through quality programs, and hi risk/low volume incidences. Once a topic is chosen and objectives are written it is time to getting writing. The faculty will provide us with the basic information and we take it from there. It is easier to write a scenario if you have access to medical records. This allows you to not come up with lab values and vital signs.

11. Nursing Education Simulation Framework (Jeffries, 2007) This theoretical framework was developed as part of the National League for Nursing and Laerdal Corp. national, multi-site, multi-method study of simulation in nursing education. I will go through these very quickly. 5 conceptual components Teacher factors-simulations are student centered Role as facilitator-provides support and encouragement We prefer this role because it is less stressful for the students. It creates a more open learning environment. Evaluator-observer We have done this role also. This caused and increase in student stress but they still performed and learned. Student factors May have them role play Student nurse, charge nurse, family member, another healthcare provider, observer, recorder. If they are required to play a role, jot down their part otherwise it can get out of hand. Response based-non active participant. Learns through observation. Process based-active participant. Need to make decisions based on patient condition. Education practices that need to be incorporated into the instruction Active engagement-limit size and time. Too large they start to wander around the room, too long they start sitting on the floor! Simulation touches visual, auditory, tactile or kinesthetic learners. Need to touch on all in the creation of your scenario. Visual-setting Auditory-verbal responses Tactile-can auscultate, take BP Kinesthetic-handling equipment, administering meds. Simulation design characteristics-this is where you will spend most of your time. Objectives Keep them simple, attainable. A early error in scenario writing occurs with too many objectives for the students to complete in the allotted time period. Need to ask-Are they really necessary or they just nice to have objectives? Fidelity-not every scenario needs a high fidelity manikin. If the objectives are all task oriented a static or low fidelity is sufficient. If the learner needs to make higher level clinical decisions-then high fidelity is more appropriate. Student support-cues given the students during the scenario. Many times we are the family member, patients or physicians voice. Debriefing-most important. Several different ways to do the debriefing. Review the objectives. Expected student outcomes Did learning occur? Were the skills performed? Was knowledge gained? Did the students express they felt more confident after the scenario?This theoretical framework was developed as part of the National League for Nursing and Laerdal Corp. national, multi-site, multi-method study of simulation in nursing education. I will go through these very quickly. 5 conceptual components Teacher factors-simulations are student centered Role as facilitator-provides support and encouragement We prefer this role because it is less stressful for the students. It creates a more open learning environment. Evaluator-observer We have done this role also. This caused and increase in student stress but they still performed and learned. Student factors May have them role play Student nurse, charge nurse, family member, another healthcare provider, observer, recorder. If they are required to play a role, jot down their part otherwise it can get out of hand. Response based-non active participant. Learns through observation. Process based-active participant. Need to make decisions based on patient condition. Education practices that need to be incorporated into the instruction Active engagement-limit size and time. Too large they start to wander around the room, too long they start sitting on the floor! Simulation touches visual, auditory, tactile or kinesthetic learners. Need to touch on all in the creation of your scenario. Visual-setting Auditory-verbal responses Tactile-can auscultate, take BP Kinesthetic-handling equipment, administering meds. Simulation design characteristics-this is where you will spend most of your time. Objectives Keep them simple, attainable. A early error in scenario writing occurs with too many objectives for the students to complete in the allotted time period. Need to ask-Are they really necessary or they just nice to have objectives? Fidelity-not every scenario needs a high fidelity manikin. If the objectives are all task oriented a static or low fidelity is sufficient. If the learner needs to make higher level clinical decisions-then high fidelity is more appropriate. Student support-cues given the students during the scenario. Many times we are the family member, patients or physicians voice. Debriefing-most important. Several different ways to do the debriefing. Review the objectives. Expected student outcomes Did learning occur? Were the skills performed? Was knowledge gained? Did the students express they felt more confident after the scenario?

12. Scenario Format Writing scenarios is a learning process. When I look back at our first attempts I am amazed how far we have come in three years. We developed a format that promotes consistency of presentation of concepts for the students. There are others in the literature or being used by other labs. This is the format we developed after facilitating scenarios for two years. We utilized the Nursing Education Simulation Framework in the development of this tool. It is also useful in the orientation of new staff. It allows staff to become more familiar the information and also provides for consistency. Explain the headings.Writing scenarios is a learning process. When I look back at our first attempts I am amazed how far we have come in three years. We developed a format that promotes consistency of presentation of concepts for the students. There are others in the literature or being used by other labs. This is the format we developed after facilitating scenarios for two years. We utilized the Nursing Education Simulation Framework in the development of this tool. It is also useful in the orientation of new staff. It allows staff to become more familiar the information and also provides for consistency. Explain the headings.

13. References: Evidence based-this is what takes up most of the time in development Use textbook for that particular class. We have all the texts in our lab for reference.References: Evidence based-this is what takes up most of the time in development Use textbook for that particular class. We have all the texts in our lab for reference.

14. Initial State: Program settings for Laerdal high fidelity manikin Patient Findings: These may be assessment data that can not be demonstrated by the manikin. May also be specific �cues� Action by Participant: Teaching Points: Information that you may want to emphasize Specifics for the facilitators. Ex: OBInitial State: Program settings for Laerdal high fidelity manikin Patient Findings: These may be assessment data that can not be demonstrated by the manikin. May also be specific �cues� Action by Participant: Teaching Points: Information that you may want to emphasize Specifics for the facilitators. Ex: OB

15. Evaluation Students Formative Summative Scenarios Feedback from students Staff review of scenarios Immediately Annually Students Formative Feedback from the facilitator 3 minute writing reflection Summative Checklists Multiple choice tests Pediatrics: pre/post test: results-increase of 13% on post-scores overall My project: pre/post test: results-increase of 11% on post-scores overall Rubric Based on Benner�s Model of Novice to Expert Wanted a score of 14 for advanced beginner Did show improvement over the semester 9.7-16.8 (average) Observation of students Journals Anecdotal notes-shows trends Scenarios Feedback from students Staff review Did the students meet the objectives? Why/why not? Updated information Realism, what can I do next time to improve? Students Formative Feedback from the facilitator 3 minute writing reflection Summative Checklists Multiple choice tests Pediatrics: pre/post test: results-increase of 13% on post-scores overall My project: pre/post test: results-increase of 11% on post-scores overall Rubric Based on Benner�s Model of Novice to Expert Wanted a score of 14 for advanced beginner Did show improvement over the semester 9.7-16.8 (average) Observation of students Journals Anecdotal notes-shows trends Scenarios Feedback from students Staff review Did the students meet the objectives? Why/why not? Updated information Realism, what can I do next time to improve?

16. Keys to Successful Simulations Keep the groups small Few objectives Use Evidence Based Practice Support from your staff in your simulation lab Practice, Practice, Practice!

17. Simulation is Fun!

18. Using High Fidelity Simulation for Annual Competency Validation of Hospital Staff  Melody Chandler, RN BSN Co-Director Regional Simulation Center Midwestern State University Another outcome from the grant is the use of high fidelity simulation to evaluate annual competency validation for approximately 900 staff of a local hospital. The way in which the hospital nurse educators and the Regional Simulation Center lab mentors worked together will be explored.Another outcome from the grant is the use of high fidelity simulation to evaluate annual competency validation for approximately 900 staff of a local hospital. The way in which the hospital nurse educators and the Regional Simulation Center lab mentors worked together will be explored.

19. History of Hospital Competency Validation Hospital staff dissatisfied with the process of repetitive competency validation through annual table top offering Hospital educators approached Regional Simulation Center (RSC) in an effort to enhance the educational offerings to the hospital staff using simulation The Regional Simulation Center (RSC) was initially established and is currently housed within the local hospital. In its evolutionary phases of development the RSC focused its first year of operations providing simulation learning to university nursing students developing scenarios based on professor driven objectives derived from course curriculum. Outcomes are based on theory from curriculum. After the first year of operations, the RSC assumed the task of evaluating annual hospital staff competencies. The process remained, as it had in the past, a year round, repetitive �table-top� offering based on requirements of the accrediting body known as The Joint Commission. The Joint Commission continuously improves the safety and quality of care provided to the public through the provision of health care accreditation and related services that support performance improvement in health care organizations. Hospital staff voiced dissatisfaction with the redundant, repetitive, uninteresting process that the institution was using to validate staff competencies. In an effort to enhance educational offerings to the hospital staff, hospital educators suggested a scenario driven annual competency validation session or �skills fair� using department specific scenarios to be delivered via high fidelity simulation.The Regional Simulation Center (RSC) was initially established and is currently housed within the local hospital. In its evolutionary phases of development the RSC focused its first year of operations providing simulation learning to university nursing students developing scenarios based on professor driven objectives derived from course curriculum. Outcomes are based on theory from curriculum. After the first year of operations, the RSC assumed the task of evaluating annual hospital staff competencies. The process remained, as it had in the past, a year round, repetitive �table-top� offering based on requirements of the accrediting body known as The Joint Commission. The Joint Commission continuously improves the safety and quality of care provided to the public through the provision of health care accreditation and related services that support performance improvement in health care organizations. Hospital staff voiced dissatisfaction with the redundant, repetitive, uninteresting process that the institution was using to validate staff competencies. In an effort to enhance educational offerings to the hospital staff, hospital educators suggested a scenario driven annual competency validation session or �skills fair� using department specific scenarios to be delivered via high fidelity simulation.

20. Initial Planning Joint Commission Accrediting body that ensures patient safety and quality of care Department Specific Competencies: Scenarios development based on department specific events and medical conditions Desired Outcomes: Hospital staff competency validation Increased participant satisfaction In the initial planning stages of the skills fair development, the institutional staff competency measures, required by the Joint Commission, were identified. Joint Commission staff competency measures: Point of care testing: BBG Restraints Crash Cart Body Mechanics Institutional staff competencies are competencies outlined by the joint commission promoting standard of care Department Specific Competencies Scenarios to be developed based on specific events and medical conditions The desired outcome being validation of annual hospital staff competencies and also increased participant satisfaction. In the initial planning stages of the skills fair development, the institutional staff competency measures, required by the Joint Commission, were identified. Joint Commission staff competency measures: Point of care testing: BBG Restraints Crash Cart Body Mechanics Institutional staff competencies are competencies outlined by the joint commission promoting standard of care Department Specific Competencies Scenarios to be developed based on specific events and medical conditions The desired outcome being validation of annual hospital staff competencies and also increased participant satisfaction.

21. Unique Perspectives Regional Simulation Staff Curriculum and theory Professor driven objectives Hospital Educators Institutional policy and procedure Each participating educational body (RSC staff and hospital educators) brought to the planning a unique perspective. Simulation staff instructional practice based on curriculum and theory, knowledge of simulation and its capacities in which learning objectives are driven by professors Hospital educators� instructional practice is based on current clinical practice governed by institutional policy and procedure with learning objectives driven by practice Employees from the RSC paired up with hospital educators and managers from individual specialty care departments to develop scenario objectives.Each participating educational body (RSC staff and hospital educators) brought to the planning a unique perspective. Simulation staff instructional practice based on curriculum and theory, knowledge of simulation and its capacities in which learning objectives are driven by professors Hospital educators� instructional practice is based on current clinical practice governed by institutional policy and procedure with learning objectives driven by practice Employees from the RSC paired up with hospital educators and managers from individual specialty care departments to develop scenario objectives.

22. Developing Department Specific Objectives Recommendations from the Joint Commission 2007 Joint Commission National Patient Safety Goals Evidence Based Practice Surviving Sepsis Campaign 5 Million Lives Saved Campaign: �How to� Guide on Preventing Ventilator Associated Pneumonia Institutional Reporting (Incident Reports) High risk low volume Low risk high volume Working together RSC staff and hospital representatives identified department specific scenario objectives based on: Recommendation from the Joint Commission 2007 National Patient Safety Goals by Joint Commission Evidence Based Practice from a review of current literature such as: 5 million lives saved campaign: How to guide on preventing Ventilator Associated Pneumonia Surviving Sepsis Campaign Institutional reporting: (Incidence Reports) High risk low volume Low risk high volume Institutional policy and procedure to mandate the appropriate actions to be performed by the hospital staff in the scenariosWorking together RSC staff and hospital representatives identified department specific scenario objectives based on: Recommendation from the Joint Commission 2007 National Patient Safety Goals by Joint Commission Evidence Based Practice from a review of current literature such as: 5 million lives saved campaign: How to guide on preventing Ventilator Associated Pneumonia Surviving Sepsis Campaign Institutional reporting: (Incidence Reports) High risk low volume Low risk high volume Institutional policy and procedure to mandate the appropriate actions to be performed by the hospital staff in the scenarios

23. Simulation: Clinical Learning and Evaluation Knowles Adult Learners bring lived experiences to the classroom Prefer self-directed learning opportunities Bandura: Social Learning Learners optimize learning through observation, imitation and modeling Theoretical underpinnings: Knowles� notes that adult learners bring multiple life experiences to the learning environment and prefer self-directed learning opportunities while being involved in the task oriented learning experience Bandura�s social learning theory holds that learners optimize learning through observation; people learn from one another, via observation, imitation, and modeling Both theories support simulation learning and evaluation of hospital staff in that employees present to the scenario with their lived experiences and work together within their peer group to complete the simulation. New horizon for evaluating competence of healthcare provider staff It is well documented that schools of nursing have benefited from the utilization of simulation however; there is very little research in respect to simulation and its value to healthcare provider competency validation. Theoretical underpinnings: Knowles� notes that adult learners bring multiple life experiences to the learning environment and prefer self-directed learning opportunities while being involved in the task oriented learning experience Bandura�s social learning theory holds that learners optimize learning through observation; people learn from one another, via observation, imitation, and modeling Both theories support simulation learning and evaluation of hospital staff in that employees present to the scenario with their lived experiences and work together within their peer group to complete the simulation. New horizon for evaluating competence of healthcare provider staff It is well documented that schools of nursing have benefited from the utilization of simulation however; there is very little research in respect to simulation and its value to healthcare provider competency validation.

24. Processes Scheduling: Participants self-scheduled into work blocks of 4 to 5 participants per group per scenario Specialty staff were divided into designated scenario timeslots prior to the skills fair Check In: Staff �checked-in� and received checklist specific to their specialty care area Checklists: Color coded for each specialty care area (ie: critical care=yellow, trauma=green) RSC staff met weekly with hospital representatives to discuss objectives and possible content to be included in each simulation. Collaboration between the two staffs maximized faculty resources and facilitated a merge from theory-based nursing education to policy-based institutional practice Hospital staff self-scheduled themselves into their work blocks of 4 to 5 participants per group per scenario Schedulers divided specialty staff into designated scenario timeslots prior to the skills fair On arrival, staff �checked-in� and received checklist specific to their specialty care area Checklist were color coded for each specialty care area (ie: critical care=yellow, trauma=green) Colored papers with unit specific competencies in a �checklist� type format were created and distributed to each participant to make it easier for participants and educators to identify what scenario the participants should be involved in for each discipline Organization was key because depending on specialty, some participants had to complete only a single simulation whereas other specialty care areas had to complete more than one. (ie: med surg had to complete only med surg, ICU had to complete critical care and pediatrics)RSC staff met weekly with hospital representatives to discuss objectives and possible content to be included in each simulation. Collaboration between the two staffs maximized faculty resources and facilitated a merge from theory-based nursing education to policy-based institutional practice Hospital staff self-scheduled themselves into their work blocks of 4 to 5 participants per group per scenario Schedulers divided specialty staff into designated scenario timeslots prior to the skills fair On arrival, staff �checked-in� and received checklist specific to their specialty care area Checklist were color coded for each specialty care area (ie: critical care=yellow, trauma=green) Colored papers with unit specific competencies in a �checklist� type format were created and distributed to each participant to make it easier for participants and educators to identify what scenario the participants should be involved in for each discipline Organization was key because depending on specialty, some participants had to complete only a single simulation whereas other specialty care areas had to complete more than one. (ie: med surg had to complete only med surg, ICU had to complete critical care and pediatrics)

25. Skills Fair Station 1: Scenarios (Individual Specialty Care Competencies) Department-specific scenarios were conducted in 30 minute intervals in department-specific scenario rooms Station 2: Skills (Institutional Staff Competencies) Completed in a central location Average time for participant completion = 1 � hours Skills fair run time: 2weeks M-F, � Saturday Mandatory completion for 900 participants Station 1: Scenarios (Individual Specialty Care Competencies) Department-specific scenarios: conducted in 30 minute intervals in department-specific scenario rooms Critical Care: Medication titration and care of patient with sepsis due to ventilator therapy Trauma Services: Pregnant patient involved in a MVA and in shock with fetal demise Medical Surgical Nursing: Basic skills involving care and maintenance of chest tube and trach Pediatrics: Nine year old with severe asthma attack requiring rapid sequence intubation Mother-Baby: Post partum Hemorrhage Nursery: Precipitous delivery with fetal demise Station 2: Skills (Institutional Staff Competencies) Completed in a central location Point of Care Testing: BBG Body Mechanics Crash Cart Restraint Safety Each participant completed the fair within and average of 1-2 hours The total run time for the skills fair was 2 weeks, 8am to 5pm Monday thru Friday and 8am to 12pm on a single Saturday. Skills fair was a mandatory educational offering for the 900 participants. Station 1: Scenarios (Individual Specialty Care Competencies) Department-specific scenarios: conducted in 30 minute intervals in department-specific scenario rooms Critical Care: Medication titration and care of patient with sepsis due to ventilator therapy Trauma Services: Pregnant patient involved in a MVA and in shock with fetal demise Medical Surgical Nursing: Basic skills involving care and maintenance of chest tube and trach Pediatrics: Nine year old with severe asthma attack requiring rapid sequence intubation Mother-Baby: Post partum Hemorrhage Nursery: Precipitous delivery with fetal demise Station 2: Skills (Institutional Staff Competencies) Completed in a central location Point of Care Testing: BBG Body Mechanics Crash Cart Restraint Safety Each participant completed the fair within and average of 1-2 hours The total run time for the skills fair was 2 weeks, 8am to 5pm Monday thru Friday and 8am to 12pm on a single Saturday. Skills fair was a mandatory educational offering for the 900 participants.

26. Skills Fair Survey Results An informal survey was randomly distributed among skills fair participants as they left the fair. This was an attempt to survey opinion about the skills fair in general. Survey results: For the overall skills fair (which would include skills stations and simulation) results were as follows: There were 197 randomly surveyed from the 900 participants: 178 thought that the fair was a good utilization of their time (time utilization), 165 felt that the content was relevant to their area (relevance) and 186 said that they learned something from the fair (knowledge acquisition).An informal survey was randomly distributed among skills fair participants as they left the fair. This was an attempt to survey opinion about the skills fair in general. Survey results: For the overall skills fair (which would include skills stations and simulation) results were as follows: There were 197 randomly surveyed from the 900 participants: 178 thought that the fair was a good utilization of their time (time utilization), 165 felt that the content was relevant to their area (relevance) and 186 said that they learned something from the fair (knowledge acquisition).

27. Simulation Survey Results Another survey was distributed to all of the participants that completed the critical care simulation. These participants would have just arrived at the fair and would have only completed simulation portion of the skills fair when surveyed. Of the 200 critical care participants surveyed, there were 184 respondents. 180 respondents said that the simulation was a good utilization of their time (time utilization), 181 found the content to be relevant to their practice area (relevance) and 180 said that they learned something from the simulation (knowledge acquisition). In addition to this survey, 160 out of 184 said that they would like to see more scenario-based simulation in the future. Skills Fair 2007 was such a success that we are already planning the next skills fair that will be held in September, 2008. Another survey was distributed to all of the participants that completed the critical care simulation. These participants would have just arrived at the fair and would have only completed simulation portion of the skills fair when surveyed. Of the 200 critical care participants surveyed, there were 184 respondents. 180 respondents said that the simulation was a good utilization of their time (time utilization), 181 found the content to be relevant to their practice area (relevance) and 180 said that they learned something from the simulation (knowledge acquisition). In addition to this survey, 160 out of 184 said that they would like to see more scenario-based simulation in the future. Skills Fair 2007 was such a success that we are already planning the next skills fair that will be held in September, 2008.

28. Outcomes of Grant 20,074 duplicated learner visits 13,444 - MSU 4,042 - VC 2,688 - URHCS 44,963 duplicated learner hours 34,116 - MSU 7742 - VC 3,105 - URHCS 900 members of URHCS Clinical Staff participated in annual competency validation through the RSC ,

29. Development of a Competency Transcript for Nurse Graduates Kathleen Roberts, RN, DNP Assistant Professor Midwestern State University Wichita Falls, Texas, USA

30. Overview Project developed in response to Nursing Redesign Taskforce of the Texas Nurses Association call for innovation and a redesign of nursing education. Plan: Develop & implement competency-based orientation process Documentation of psychomotor & decision making skills given to employer Accepted as graduate�s �credentials� Used to determine orientation needs of graduate Nursing Redesign Task force of TNA TNA recommended developing methods to: -- �Achieve common language between education and practice� -- �Create new methods to assess, track and document clinical competencies of students.� Develop & field competency transcript to document mastery of medical-surgical competencies of BSN graduates - Nursing Redesign Task force of TNA TNA recommended developing methods to: -- �Achieve common language between education and practice� -- �Create new methods to assess, track and document clinical competencies of students.� Develop & field competency transcript to document mastery of medical-surgical competencies of BSN graduates -

31. Background 57% nurse administrators � BSN grads do not meet competency expectations Only 35% nurse graduates meet entry level expectations of clinical judgment Orientation � 7 weeks Costs $15,000 - $50,000 55-61% turnover of grad nurses CNOs believe orientation can be shortened 10-50% Texas � Besides literature already discussed Nursing school admission and graduation rates are increasing, yet demand still exceeds potential supply (THECB) Competency Literature 57% nurse administrators surveyed � BSN graduates did not meet their competency expectations (interpersonal communication, health promotion, direct care DelBueno � 35% met entry level expectations for clinical judgment Orientation Grad nurses � Average 7 weeks � range from days to 96 months Graduate nurses describe too many didactic courses become overwhelming Nurse administrators � despite extensive orientations: 1. 55-61% turnover of grad nurses and 2. cite lack of critical thinking and inability to function in interdisciplinary team Many CNO � agree competency testing could potentially reduce lengthy classroom orientation � especially if learner could test out - May save 10 � 50% orientation time! Competency Based � Used for years with seasoned nurses When focus of new grads� orientation is to reteach everything � graduates confidence is diluted. Focus on limitations puts emphasis on the right things � leading to confidence, retention of knowledge and competence Cost - $15,000 - $50,000 Texas � Besides literature already discussed Nursing school admission and graduation rates are increasing, yet demand still exceeds potential supply (THECB) Competency Literature 57% nurse administrators surveyed � BSN graduates did not meet their competency expectations (interpersonal communication, health promotion, direct care DelBueno � 35% met entry level expectations for clinical judgment Orientation Grad nurses � Average 7 weeks � range from days to 96 months Graduate nurses describe too many didactic courses become overwhelming Nurse administrators � despite extensive orientations: 1. 55-61% turnover of grad nurses and 2. cite lack of critical thinking and inability to function in interdisciplinary team Many CNO � agree competency testing could potentially reduce lengthy classroom orientation � especially if learner could test out - May save 10 � 50% orientation time! Competency Based � Used for years with seasoned nurses When focus of new grads� orientation is to reteach everything � graduates confidence is diluted. Focus on limitations puts emphasis on the right things � leading to confidence, retention of knowledge and competence Cost - $15,000 - $50,000

32. Problem No user-friendly methodology to describe the nursing competencies of new grads to their employers Apprentice-like mindset discounts professional preparation; RNs required to re-enter training to practice their �craft� Given what has been happening in Texas and described in the nursing literature: Evidence of preparation to practice: RN licensure & college transcripts 2. Extensive orientations � re-teach many skills and proceduresGiven what has been happening in Texas and described in the nursing literature: Evidence of preparation to practice: RN licensure & college transcripts 2. Extensive orientations � re-teach many skills and procedures

33. Goal Develop & implement a competency based orientation process Create documentation process using �competency transcript� (CT) Psychomotor skills Clinical judgment ability Accepted as graduate�s credentials for practice Employers train to the gap Documents mastery of medical-surgical competencies - Foundation from which orientation is built Documents mastery of medical-surgical competencies - Foundation from which orientation is built

34. Theoretical Framework Patricia Benner�s (1984) �Novice to Expert� Used two of her developmental levels; created a third �Unsafe�

35. Scenario-Based Decision Making 12 patient care situations Same scenarios practiced in regional simulation center (RCS)

36. Reliability and Validity Established content and construct validity Content � 83% items �quite relevant�; 17% �very relevant� Construct � RNs scored significantly higher than students in 11 of 12 scenarios Established intra-rater reliability 92% agreement of scores 3 months apart Three med/surg experts rated each line item in rubric � 83% �quite relevant�; 17% �very relevant� on 4 level scale (not relevant, somewhat relevant, quite relevant or very relevant) Construct � Predicted RNs with 2-3 years of experience would score as Advanced Beginners 8 RNs responded to same 12 scenarios as 46 senior nursing students did RNs scored significantly higher than students in 11 of 12 scenariosThree med/surg experts rated each line item in rubric � 83% �quite relevant�; 17% �very relevant� on 4 level scale (not relevant, somewhat relevant, quite relevant or very relevant) Construct � Predicted RNs with 2-3 years of experience would score as Advanced Beginners 8 RNs responded to same 12 scenarios as 46 senior nursing students did RNs scored significantly higher than students in 11 of 12 scenarios

37. Construct Validity

38. Construct Validity Scenario #3 � Smoking cessation/DVT scenario - Both groups were novices in the way they interacted with the patient Most did not include smoking cessation considerations in their responses. � Even though this has been a focus in education and service (hospitals) because of the Agency for Healthcare Research and Quality (AHRQ) since 2003. Conflict 1 � Heparin and blood glucose Conflict 2 � Blood transfusion and med administration (Rocephin) Conflict 3 � MD phones with orders � chest pain Discharge planningScenario #3 � Smoking cessation/DVT scenario - Both groups were novices in the way they interacted with the patient Most did not include smoking cessation considerations in their responses. � Even though this has been a focus in education and service (hospitals) because of the Agency for Healthcare Research and Quality (AHRQ) since 2003. Conflict 1 � Heparin and blood glucose Conflict 2 � Blood transfusion and med administration (Rocephin) Conflict 3 � MD phones with orders � chest pain Discharge planning

39. Outcome Measures Orientation time decreased 25% Fewer items trained for RNs with CT Cost of orientation decreased 10% RNs with CT are retained longer Employers trust CT Orientation time - # days in orientation; graduates with/without CT t-test, descriptive statistics Cost of orientation � Cost of orientation per orientee � per day -- unit specific -- hospital Retention of RNS with Ct - turnover rates - Recruitment costs/orientee Employers trust CT Likert survey � given before/after implementation of CT --elicits employer�s understanding of CT, its purpose as well as trust in CT accuracyOrientation time - # days in orientation; graduates with/without CT t-test, descriptive statistics Cost of orientation � Cost of orientation per orientee � per day -- unit specific -- hospital Retention of RNS with Ct - turnover rates - Recruitment costs/orientee Employers trust CT Likert survey � given before/after implementation of CT --elicits employer�s understanding of CT, its purpose as well as trust in CT accuracy

40. Results Orientation time with CT longer than non-CT No difference in orientation costs No data available to evaluate other measures

41. Other Findings Graduates score as �Novice� - different than Benner�s findings Differences in complexity of care Benner recommended using �Beginner� Results from critical thinking tool Applicable to educators and service Educators don�t know how to use clinical judgment information Clinical judgment rubric difficult to use Benner, P. (2008, May 2), Personal discussion at TNCC Invitational Conference, Lubbock Texas. Exploring the relationship between apprenticeships and ongoing competencies Benner, P. (2008, May 2), Personal discussion at TNCC Invitational Conference, Lubbock Texas. Exploring the relationship between apprenticeships and ongoing competencies

42. Recommendations Implement transcript at pilot test site Larger sample Automate clinical judgment rubric Provide guidance in use of clinical judgment information Preceptor training Implementation tools

43. High Fidelity Patient Simulation:Cost Effectiveness Susan Sportsman, RN, Ph.D., Dean Midwestern State University College of Health Sciences and Human Services

44. Cost Effectiveness Cost-Benefit Analysis (CBA) Traditional framework in which the $ value of net social benefits is compared to $ value of the project.

45. Cost Effectiveness Cost Effectiveness Discounted cash-flow analysis (DCF) Organization-centered, focusing on the extent to which the project can be funded through cash flow and cash savings

46. Reasons for choosing DCF framework Although project was initiated through a grant, should ultimately be self sufficient Provides a framework to assess financial viability of such an assessment for other organizations

47. Objective of DCF To determine if the investment costs for setting up the patient simulation center are offset by the net savings and additional cash in-flows that may derive from using the simulation center rather than school or hospital-specific lab approach

48. Costs of the Project Investment cost-simulation equipment Staff time to set-up lab and develop scenarios Faculty costs savings RSC staff costs

49. Results There are substantial savings in instructional costs that should occur as a result of the use of patient simulators. The savings are not sufficient to offset the investment costs based on the study assumptions. Harlow, K. & Sportsman, S. An economic analysis of patient simulators for clinical training in nursing education. Nursing Economics. 25(1) January, 2007.

50. Qualitative Evaluation: Focus Groups

51. Quantitative Data/Results Three Year Data Collection Process January 2005, 2006, & 2007: Juniors April 2005,2006 & 2007: Seniors 2005 Seniors - Little or no simulation experience 2005 Juniors/2006 Seniors - 3 semesters of simulation experience 2006 Juniors/2007 Seniors - 5 semesters of simulation experience

52. Data Collection Instruments Clinical Competence Appraisal Scale (PSP, Leadership, Teaching/Collaboration, Interpersonal Relations/Communication, Planning/Evaluation) LASSI (Motivation, Attitude, Concentration, Anxiety) Clinical Learning Environment Demographic Data Sheet GPA Scores on HESI

53. Conclusions 2005 juniors (no simulation) rated their PSP performance significantly higher (p=.0001) than juniors in 2006 and 2007. Participating in simulation early in their clinical experience may provide a �dose of realism� for students in their clinical courses. ________________ Substituting clinical experience in the RSC for a portion of the time required in clinical agencies does not make a difference in students� perception of their clinical competence. Argues for the substitution of simulation experience for some clinical experiences as a strategy for increasing student admissions when there are limited clinical experiences available to schools

54. Conclusions 2005 seniors had a significantly higher mean anxiety score (p=.015) (less anxiety) than the seniors in 2006 and 2007 Increased participation in simulation experiences may have contributed to the increase in the 2006 and 2007 senior students� anxiety. ------------------------------- The mean scores on the CLE Scale for the 2005 seniors were significantly lower than the mean scores for seniors in 2006 and 2007. Simulation may positively influence students� perceptions of the clinical environment where they are assigned during the last semester of their course of study.

55. Conclusions No significant difference in graduating GPA or HESI E2 Exit Exam for seniors in 2005, 2006 and 2007 Participation in scenario-based simulation does not negatively impact performance on the HESI exam. (Results highly correlated with success on the NCLEX-RN licensing exam (Morrison, et. al, 2004).

56. After the Grant Funding Based on Usage

57. MOU for Consortium The North Central Texas Health Care Consortium (NCTHCC) Board is responsible for: Selection and evaluation of the Director of the RSC Determination of overall policies regarding the RSC Financial management of RSC funds, including budget development Determination of future research or program development Consortium Members Dean of the College of HSHS, MSU Chair, MSU Nursing Director, VC Program Senior VC faculty member from VC Vice President , Patient Care Services, URHCS Manager of Education, URHCS Director (s) of RSC.

58. The Consortium The Chair - senior representative of the organization who serves as the financial manager. All decisions of the NCTHCC are made through a simple majority. Member organizations responsible for operational costs of the RSC based upon the percentage of time learners partner spent in RSC the previous academic year. The NCTHCC Board will meet at least four times a year.

59. QUESTIONS?