pieces of the puzzle n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Pieces of the Puzzle PowerPoint Presentation
Download Presentation
Pieces of the Puzzle

Loading in 2 Seconds...

play fullscreen
1 / 26

Pieces of the Puzzle - PowerPoint PPT Presentation


  • 176 Views
  • Uploaded on

Pieces of the Puzzle. Perspectives on the Accreditation Visit……. Central Theme. Accreditation and Assessment are critically linked in contemporary accreditation models. They were parallel themes in recent past but have “collided” to form a new accountability in higher education.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Pieces of the Puzzle' - calida


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
pieces of the puzzle

Pieces of the Puzzle

Perspectives on the Accreditation Visit……

central theme
Central Theme
  • Accreditation and Assessment are critically linked in contemporary accreditation models.
  • They were parallel themes in recent past but have “collided” to form a new accountability in higher education.

Ratcliff JL et al, How Accreditation

Influences Assessment, Josey Bass 2001

how does accreditation and assessment come together
How does accreditation and assessment come together?
  • Two key questions:
    • How should learner outcomes be demonstrated in the accreditation process?
    • How should assessment information be used to show improvement in programs, services, and student/resident learning?
accreditation purposes
Accreditation Purposes
  • Fostering excellence through the development of criteria and guidelines for assessing effectiveness
  • Encouraging improvement through ongoing self-study and planning
  • Ensuring external constituents that a program has clearly defined goals and appropriate objectives, maintains faculty and facilities to attain them, demonstrates it is accomplishing them, and has the prospect for continuing to do so

Ratcliff JL et al, How Accreditation

Influences Assessment, Josey Bass 2001

accreditation purposes1
Accreditation Purposes
  • Provides advice and counsel to new and established programs in the accrediting process
  • Ensures that programs receive sufficient support and are free from external influence that may impede their effectiveness and the freedom of inquiry

Ratcliff JL et al, How Accreditation

Influences Assessment, Josey Bass 2001

program accreditation
Program Accreditation
  • Needs to fulfill two dimensions of program quality:
    • There should be some evidence that the programs have clear goals and courses of study to attain them
    • The process should demonstrate that responsibilities associated with the goals are being carried out.
  • Peers interpret quality in the context of the program’s own aims and activities
    • Central issue: Program review is conducted as a self study process.
    • Central assumption: Evaluators from outside cannot effectively examine and judge until those inside have reached a judgment about its quality.

Ratcliff JL et al, How Accreditation

Influences Assessment, Josey Bass 2001

program review audit style
Program Review – Audit Style
  • Issues identified by “outside party”
  • Buy-in from program director?
    • Reactive –
      • Fear about the inspection
    • Approach – “cover the basics, get by this one and on to the next”
    • Not stimulated to think about “points of excellence” – only points of deficiency
  • Outcome from Inspection:
    • Fear relieved – “get on to the next one.”
  • Yardstick: STANDARDS – Basic/Minimal
program review assessment style
Program Review Assessment Style
  • Self Study and Peer assessment against target goals:
    • Minimal standards = musts
    • Growth goals
    • Improvement goals
    • Outcome goals
  • Buy-in for Program Director
    • Proactive – ownership of goals
  • Outcome of Inspection
    • External validation of internal assessment (agree or not)
    • How much progress has been made against goals?
  • Yardstick: Goals achieved or improvement beyond minimal standards – Points of Excellence
slide9

Practicality?

What about the Residents?

OPTI Standards

Basic Standards

For

I R

Time?

Last

Inspection?

Family?

How am I

Doing as a PD?

Patient Care?

Federal and Local Rules

Inspector Factor ?

Specialty College

Standards

Productivity?

Salary?

Malpractice?

slide10

What am I supposed

To do with this?

OPTI Standards

Basic Standards

For

I R

???

???

YES!!

What role?

Federal and Local Rules

Specialty College

Standards

slide11

?

OPTI Standards

Do they agree???

Basic Standards

For

I R

Which one takes

precedence?

?

?

Federal and Local Rules

?

Specialty College

Standards

experience with the audit program review
Experience with the Audit Program Review….
  • Attempts to help program directors and DMEs to prepare for upcoming inspections.
  • Attempts to find “common areas” of support need across OPTI programs of same discipline
  • Based on BASIC STANDARDS – developed “Crosswalks” to assist in review of individual standards and collective standards.
slide13

Standards Crosswalk and Worksheet

Completed by:_____________

Date:_____

Specialty:______________

slide14

Program Pre-Inspection Findings

Hospital_______________

Completed by:________________

standards often do not
Standards often do not:
  • Consider practical issues
    • A complete research paper per year of residency (human subject approvals take almost half that time).
    • Ability of community hospital to afford “out rotations” since the BBA 1997
    • Ability of trainer to (again) teach subjects which were dropped from practice in time past.
  • Consider federal rules or procedures
    • Institutional Review Board and HIPPA approval for human subject research
    • Documentation standards
standards often do not1
Standards often do not:
  • Reflect the need to include the following in the curriculum and assess effectiveness:
    • Ethics
    • Communication skills
  • Require the assessment of outcomes of study
    • Clinical skill development or OSCEs
    • In Service Examination and Analysis (individual and program wide).
    • Progress exams or oral exams
  • Require that a program develop an ongoing quality assessment program and benchmarks for process and learning outcomes goals.
experiences with accreditation visits an opti view example
Program Review

40 “areas of concern” – standards not met in crosswalk

Affiliation agreements for major components not completed (for over 10 years)

Manual in disarray…never distributed to residents for past three years

PD mostly practices at alternate hospitals (where residents not welcome) for >70% time

Etc.

Inspector

“Wonderful program with dedicated PD and hospital”

No major deficiencies

Recommend increase in number of trainees

Commendable PD attention to program

Recommend full five year accreditation.

Experiences with Accreditation Visits – an OPTI view ..example:

Net result:: “OPTI is too tough….

see, We are OK!!”

experiences with accreditation visits an opti view example1
Program Review

36 “areas of concern” on Standards Crosswalk

No resident manual in print/no written curriculum

Resident work hours = 2 weeks on call continuously

2 yr program required by PD/1 yr by specialty college….few recent trainees complete the two years. Certificates difficult to obtain.

Certification rates = <%50 pass rate on first attempt.

Inspector exit notes:

“Excellent program which should be a model for others”

This program should be expanded to train more residents.

No major deficiencies

Experiences with Accreditation Visits – an OPTI view ..example:

Net result:: PD who had been working with OPTI to correct findings has stopped. “We are doing fine….see how well we did on the inspection? Why won’t you sponsor more positions?”

experiences with accreditation visits the inspector factor
Experiences with Accreditation Visits – the “Inspector Factor”
  • Inspectors
    • Often not familiar with Basic Standards for Interns and Residents or OPTI standards
    • Often spend as little as 4-6 hours in the entire process
    • Do not “dig deep enough” to see issues
    • Accept inspectors workbook without verifying content.
  • Inspectors workbooks and standards often do not reflect the same “intent” or priority for the issue in question – produce confusion
    • Ranking of standards is evident by threat of automatic probation if question answered “NO”.
  • “Inter-rater reliability” seems low
conclusion audit review
Conclusion – Audit Review
  • “Dissatisfaction” with end result
    • Gets through the current “audit” inspector approach
    • Fails to stimulate long term goals or real improvement
    • Something is missing
  • May promote a punitive rather than a supportive atmosphere –
begin with the end in mind what is the goal
“Begin with the end in mind”- What is the goal?
  • Get from one accreditation visit to the next?
    • Minimal level thinking
    • Threshold is the lowest possible point to keep the program afloat (?)
  • Continuous improvement over time against declared goals.
    • Determine a base line
    • Set realistic but challenging goals
    • Measure progress….set new goals
    • (meeting minimal thresholds is never a problem)
goal get from one accreditation visit to the next
Goal –get from one accreditation visit to the next?
  • Data driven
  • Minimalist thinking
  • “audit mentality”
  • Promotes bashing and blaming
    • Sets interested parties at odds
  • Promotes a pessimistic environment and inhibits creative thinking
  • Goals are at the minimal threshold – “Basic”
goal assessment for cqi and outcomes
Goal – Assessment for CQI and Outcomes
  • Markers of Excellence
  • “Internalized goals” born in self study and assessment
  • “Buy-in” for all stakeholders
goals for the workshop
Goals for the Workshop
  • Recreate the “Basic Standards” to allow and encourage growth, goal setting and a reach for higher goals.
  • Revise and recreate basic standards to ensure basic elements are present and to foster agreement between:
    • External standards/Internal standards
    • External validation/Internal validation
  • To foster better understanding of the “fit” between the AOA standards and laws which apply to residency training.
slide25

Basic Standards

For

I R

OPTI Standards

Federal and Local Rules

Specialty College

Standards

Goal