Principal evaluation
Download
1 / 82

Principal Evaluation - PowerPoint PPT Presentation


  • 75 Views
  • Uploaded on

Principal Evaluation. August 2012 Iowa Joseph Murphy Vanderbilt University. IOWA August 2012. State & district policy. Standards. District & state policy. evaluation. Conditions of Work. governance. incentives. professional development. l i c e n s u r e. Career Line.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Principal Evaluation' - laura-vincent


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Principal evaluation

Principal Evaluation

August 2012

Iowa

Joseph Murphy

Vanderbilt University



State & district policy

Standards

District & state policy

evaluation

Conditions of Work

governance

incentives

professional development

l

i

c

e

n

s

u

r

e

Career Line

preparation

internship

induction (residency)

accreditation/program approval

professional development

teacher leadership preparation

relicensure

mentoring

District &

state policy

Education

State & district policy

Standards


LEVERAGE POINT MODEL

National

Iowa

data

data


Think pair share
Think, Pair, Share

  • Describe the current evaluation system in your district.

  • List two things that you don’t like about it.

  • On a scale of 1 (not useful) to 10 (very useful),

    how useful is it in:

     showing you who your really strong principals are

     ensuring principal accountability

     providing targeted professional growth for principals

     promoting improvement at each school

     promoting district-level improvement initiatives


Principal evaluation the national story

Principal Evaluation: The National Story


Principal evaluation the national story1
PRINCIPAL EVALUATION: The National Story

PART A:

Problems with Current Evaluation Designs

PART B:

“Construction Crew” Work in Other States


Part a
PART A:

Problems with Current Evaluation Systems



Content
Content

  • Little evidence that systems evaluate what is important, i.e. not valid

  • Insufficient attention to leadership for learning, especially curriculum and instruction


Process
Process

  • Limited architecture – focus on one approach (e.g. goals)

  • Perfunctory – not a deep process


Impact
Impact

  • Principals not receiving useful feedback

  • Not promoting professional growth of principals

  • Not promoting organizational improvement


Part b
PART B:

Construction Crew Work


Principal evaluation construction crew work
Principal Evaluation“Construction Crew” Work



Leadership
Leadership

1. Leadership matters.

2. In difficult times, leadership matters even more.

3. In periods of significant organizational transition, leadership is the major controllable factor in explaining organizational performance.


Leadership1
Leadership

4. Instructionally-focused and change-

oriented leadership are especially

effective frames for education.

5. Team leadership seems to offer promise

for enhancing organizational performance.

6. Assessment provides a strategic leverage

point for strengthening leadership.


School

Experience

  • Accountability

  • External conditions

  • Standards

  • Curriculum

  • Instruction

  • Culture

Value-added

Distribution

Level

Knowledge

Personal

Characteristics

Leadership

Behaviors

Student

Success

  • Achievement

  • Graduation

  • College attendance

  • Post graduation success

Values &

Beliefs

Classroom

“Context”

  • School

  • Type

  • Level

  • Nature

District & State

  • Student composition

  • Staff composition

“precursors”

“’behaviors”

“influence pathway”

“’outcomes”

Figure 1. Learning Centered Leadership Framework


Table 1 knowledge base for the assessment system
Table 1– Knowledge Base for the Assessment System

I. Vision for Learning

II. Instructional Program

III. Curricular Program

IV. Assessment Program

V. Communities of Learning

VI. Resource Acquisition and Use

VII. Organizational Culture

VIII. Social Advocacy


I vision for learning
I. Vision for Learning

A. Developing vision

B. Articulating vision

C. Implementing vision

D. Stewarding vision


Ii instructional program
II. Instructional Program

A. Knowledge and involvement

B. Hiring and allocating staff

C. Supporting staff

D. Instructional time


Iii curricular program
III. Curricular Program

A. Knowledge and involvement

B. Expectations, standards

C. Opportunity to learn

D. Curriculum alignment


Iv assessment program
IV. Assessment Program

A. Knowledge and involvement

B. Assessment procedures

C. Monitoring instruction and

curriculum

D. Communication and use of data


V communities of learning
V. Communities of Learning

A. Professional development

B. Communities of professional

practice

C. Community-anchored schools


Vi resource acquisition and use
VI. Resource Acquisition and Use

A. Acquiring resources

B. Allocating resources

C. Using resources


Vii organizational culture
VII. Organizational Culture

A. Production emphasis

B. Learning environment

C. Personalized environment

D. Continuous improvement


Viii social advocacy
VIII. Social Advocacy

A. Stakeholder engagement

B. Diversity

C. Environmental context

D. Ethics


ISLLC

Educational Leadership Policy Standards: 2008

As Adopted by the National Policy Board for Educational Administration


STANDARD 1: An education leader promotes the success of every student by facilitating the development, articulation, implementation and stewardship of a vision of learning that is shared and supported by all stakeholders

Functions:

  • A. Collaboratively develop and implement a shared vision and mission

  • B. Collect and use data to identify goals, assess organizational effectiveness, and promote

  • organizational learning

  • C. Create and implement plans to achieve goals

  • D. Promote continuous and sustainable improvement

  • E. Monitor and evaluate progress and revise plans


STANDARD 2:An education leader promotes the success of every student by advocating, nurturing, and sustaining a school culture and instructional program conducive to student learning and staff professional growth.

  • Functions:

  • A. Nurture and sustain a culture of collaboration, trust, learning, and high expectations

  • B. Create a comprehensive, rigorous, and coherent curricular program

  • C. Create a personalized and motivating learning environment for students

  • D. Supervise instruction

  • E. Develop assessment and accountability systems to monitor student progress

  • F. Develop the instructional and leadership capacity of staff

  • G. Maximize time spent on quality instruction

  • H. Promote the use of the most effective and appropriate technologies to support teaching

  • and learning

  • I. Monitor and evaluate the impact of the instructional program


STANDARD 3: An education leader promotes the success of every student by ensuring management of the organization, operation, and resources for a safe, efficient, and effective learning environment.

Functions:

  • A. Monitor and evaluate the management and operational systems

  • B. Obtain, allocate, align, and efficiently utilize human, fiscal, and technological resources

  • C. Promote and protect the welfare and safety of students and staff

  • D. Develop the capacity for distributed leadership

  • E. Ensure teacher and organizational time is focused to support quality instruction and

  • student learning


STANDARD 4: An education leader promotes the success of every student by collaborating with faculty and community members, responding to diverse community interests and needs, and mobilizing community resources.

Functions:

  • Collect and analyze data and information pertinent to the educational environment

  • B. Promote understanding, appreciation, and use of the community’s diverse cultural, social,

  • and intellectual resources

  • C. Build and sustain positive relationships with families and caregivers

  • D. Build and sustain productive relationships with community partners


STANDARD 5: An education leader promotes the success of every student by acting with integrity, fairness, and in an ethical manner.

Functions:

  • A. Ensure a system of accountability for every student’s academic and social success

  • B. Model principles of self-awareness, reflective practice, transparency, and ethical behavior

  • C. Safeguard the values of democracy, equity, and diversity

  • D. Consider and evaluate the potential moral and legal consequences of decision-making

  • E. Promote social justice and ensure that individual student needs inform all aspects of

    schooling


STANDARD 6: An education leader promotes the success of every student by understanding, responding to, and influencing the political, social, economic, legal, and cultural context.

Functions:

  • A. Advocate for children, families, and caregivers

  • B. Act to influence local, district, state, and national decisions affecting student learning

  • C. Assess, analyze, and anticipate emerging trends and initiatives in order to adapt

  • leadership strategies


Metropolitan Institute for Leadership in Education (MILE) and the GSU Teacher Quality Partnership (TQP) GrantDeveloped by Southland Administrators in Consultation with Dr. Joseph Murphy, Vanderbilt University and aligned with state of Illinois PERA 2010 Initiative Coordinator, Dr. Donna Joy

Principal Performance-Based Evaluation Prototype 2011-2012


Guiding principles of the evaluation system
Guiding Principles of the Evaluation System and the GSU Teacher Quality Partnership (TQP)

Foundations

  • Highlight teaching and learning

  • Include student growth as a significant factor

  • Be standards based: ISLLC and Illinois Performance Standards for School Leaders

  • Underscore learning-centered leadership


Guiding principles of the evaluation system1
Guiding Principles of the Evaluation System and the GSU Teacher Quality Partnership (TQP)

Process

● Be evidence based

● Have set benchmarks agreed upon in advance 

● Be transparent

● Foster a culture of collaboration between the principal and the supervisor

● Be valid and reliable

● Be comprehensive but not overly complex

● Be both formative and summative

● Include multiple measures

● Tap into multiple constituents

● Have well-defined timelines

● Provide ongoing feedback to the principal

● Be site specific, connected to the needs of the specific school

● Be flexible enough to allow for adjustments


Guiding principles of the evaluation system2
Guiding Principles of the Evaluation System and the GSU Teacher Quality Partnership (TQP)

Outcomes

  • Motivate principals to improve

  • Promote targeted professional growth opportunities

  • Promote school improvement

  • Enhance academic and social learning of students

  • Result in positive or negative consequences


Components of the evaluation system
Components of the Evaluation System and the GSU Teacher Quality Partnership (TQP)

  • VAL-ED – 25%

    Assessment on ISLLC Standards/Illinois Performance Standards for School Leadership

  • Organizational and Professional Goals (25%)

    Assessment on meeting an organizational goal (5-20%)

    Assessment on meeting a professional development goal (5-20%)

  • Student Growth (50%)

    Assessment on growth in student achievement


Components of the evaluation system1
Components of the Evaluation System and the GSU Teacher Quality Partnership (TQP)

Component 1 (25%):

  • 360 degree assessment of principal’s leadership on

    instructional leadership

  • Provides nationally normed measures of principal performance

  • In-depth discussion of VAL-ED later this morning


Components of the evaluation system2
Components of the Evaluation System and the GSU Teacher Quality Partnership (TQP)

Components 2 & 3

Organizational and Growth Goals – One each – 25% total

Source of goals:

  • Student outcomes

  • District and school improvement plans

  • Board of education and superintendent objectives

  • Surveys (e.g., of school climate, parent satisfaction, and so forth)

  • Previous evaluations (i.e., data from assessments of components in the evaluation system)

  • Accreditation reports


Components of the evaluation system3
Components of the Evaluation System and the GSU Teacher Quality Partnership (TQP)

Components 2 & 3

Organizational and Growth Goals – One each – 25% total

Characteristics of Effective Goals:

  • Be linked to the Illinois Performance Standards for Schools Leaders and ISLLC Standards.

  • Be organizationally grounded and emphasize the direct contributions of the leader.

  • Be anchored in an analysis of multiple sources around relevant data.

  • Be specific, measurable, attainable, realistic, timely, and challenging.

  • Have a longitudinal focus.

  • Be mutually determined through collaborative dialogue.

  • Be collaboratively reviewed with frequent and specific feedback.


Components of the evaluation system4
Components of the Evaluation System and the GSU Teacher Quality Partnership (TQP)

Growth In Student Achievement – 50%

Source of Goals:

  • State norm-referenced and criterion-referenced tests

  • Promotion and high school completion

  • RTI/progress monitoring data

  • College entrance examinations

  • End-of-course examinations

  • Advanced placement examinations

  • Other local assessments


The evaluation process
The Evaluation Process and the GSU Teacher Quality Partnership (TQP)


The evaluation process1
The Evaluation Process and the GSU Teacher Quality Partnership (TQP)

Goal Setting(by June 30)

Steps:

  • Review data sources prior to meeting

  • Bring relevant data to the meeting

  • Discuss and agree on:

    • Goals

    • Sources of evidence

    • Performance measures


The evaluation process2
The Evaluation Process and the GSU Teacher Quality Partnership (TQP)

Formative Conference: (by November 30)

Steps:

  • Examine evidence and discuss the progress on target

    measures.

  • Review any new data available.

  • Make adjustments to the goals as necessary.

  • Add resources and supports to make goals attainable.


The evaluation process3
The Evaluation Process and the GSU Teacher Quality Partnership (TQP)

VAL-ED Assessment (In December)

Completion of the VAL-ED assessment by the supervisor, principal and teachers in December.


The evaluation process4
The Evaluation Process and the GSU Teacher Quality Partnership (TQP)

Principal’s Pre-Summative Self-Evaluation

(by February 1)

Steps:

  • Gather evidence

  • Reflect on progress

  • Provide written self-assessment


The evaluation process5
The Evaluation Process and the GSU Teacher Quality Partnership (TQP)

Summative Evaluation (by March 1)

Steps:

  • Meet and discuss self-reflection of principal

  • Evaluator shares her/his perspective

  • Sign off, using scoring system


Scoring system components
SCORING SYSTEM and the GSU Teacher Quality Partnership (TQP) Components

VAL-ED Performance (25%)

Basic

Proficient

Below Basic

Distinguished

1

Does Not Meet

Expectations

2

Basic

3

Proficient

4

Exceeds

Expectations


Scoring system components1
SCORING SYSTEM and the GSU Teacher Quality Partnership (TQP) Components

Organizational Performance Goal (5% to 20%)

Approaches the target

Meets the target

Misses the target by a considerable margin

Significantly exceeds the target

1

Does Not Meet

Expectations

2

Basic

3

Proficient

4

Exceeds

Expectations


Scoring system components2
SCORING SYSTEM Components and the GSU Teacher Quality Partnership (TQP)

Personal Growth Goal (5% to 20%)

Approaches the target

Meets the target

Misses the target by a considerable margin

Significantly exceeds the target

1

Does Not Meet

Expectations

2

Basic

3

Proficient

4

Exceeds

Expectations


Scoring system components3
SCORING SYSTEM Components and the GSU Teacher Quality Partnership (TQP)

Student Achievement Goal(value-added) (50% )

Approaches the target

Meets the target

Misses the target by a considerable margin

Significantly exceeds the target

1

Does Not Meet

Expectations

2

Basic

3

Proficient

4

Exceeds

Expectations


Composite rating 250 100 2 5
Composite Rating 250/100 2.5 and the GSU Teacher Quality Partnership (TQP)

Weights (sample scoring)


Final evaluation rating
Final Evaluation Rating and the GSU Teacher Quality Partnership (TQP)

Unsatisfactory 1.0-1.6

Needs Improvement 1.7-2.4

Proficient 2.5-3.2

Excellent 3.3-4.0

The categories above are required by PERA

The numbers above are an initial cut; they may need refinement.


I vanderbilt assessment of leadership in education val ed
I. Vanderbilt Assessment of Leadership in Education and the GSU Teacher Quality Partnership (TQP) VAL-ED


The val ed instrument
The VAL-ED Instrument and the GSU Teacher Quality Partnership (TQP)

The instrument consists of 72 items defining six core component subscales and six key process subscales.

Principal, Teachers, & Supervisor provide a 360-degree, evidenced-based assessment of leadership behaviors.

Respondents rate effectiveness of 72 behaviors on scale 1=Ineffective to 5=Outstandingly effective.

Each respondent rates the principal’s effectiveness after indicating the sources of evidence on which the effectiveness is rated.

Two parallel forms of the assessment facilitate measuring growth over time.

The instrument is available in both paper and online versions.


Purpose uses
Purpose & Uses and the GSU Teacher Quality Partnership (TQP)

The VAL-ED can be used as part of a comprehensive assessment of the effectiveness of a leader's behaviors.

The VAL-ED reports principal performance through

Norm-referenced scores and

Criterion-reference scores.

VAL-ED can be used annually or more frequently to:

Facilitate a data-based performance evaluation,

Measure performance growth, and

Guide professional development.


Implementation
Implementation and the GSU Teacher Quality Partnership (TQP)

Identify respondents and invite participation.

Discuss use of results & confidentiality.

Decide paper or online version.

Time and Timing

Average respondent requires 20 to 25 minutes.

Schedule completion after respondents have had a reasonable time to observe/experience the leader’s work and its effects on the school.

Designate person(s) to manage collection and submission of response forms, if paper version used.

Ensure teacher confidentiality.


Directions for completing rating scale
Directions for Completing Rating Scale and the GSU Teacher Quality Partnership (TQP)


An example set of responses
An Example Set of Responses and the GSU Teacher Quality Partnership (TQP)



Assessing learning centered leadership the val ed vision
Assessing Learning-Centered Leadership: PropertiesThe VAL-ED vision…

A leadership assessment system that has the following properties:

Works well in a variety of settings and circumstances,

Is construct valid,

Is reliable,

If feasible for widespread use,

Provides accurate and useful reporting of results.

Is unbiased,

Yields a diagnostic profile for summative and formative purposes.

Can be used to measure progress over time in the development of leadership, and

Predicts important outcomes.


Psychometric evidence
Psychometric Evidence Properties

Item and response scale development

Based on review of learning-centered leadership literature and alignment to ISLLC standards.

Critiqued by education leaders and leadership researchers.

Item sorting study

Established content validity by asking education leaders to sort the items into 36 cells.

Cognitive interviews of paper/pencil version

Two rounds of cognitive interviews in three districts each.

Three respondents evaluated the format and items.

Nine-school pilot test (320/440 teachers = 73%)

Estimated reliability of each of 12 scales.

Established construct validity through factor analysis.

Established face validity through questions to respondents.


Psychometric evidence1
Psychometric Evidence Properties

Cognitive interviews of online instrument including revisions based on 9-school pilot

Bias review

Submitted to urban districts to evaluate language.

11-school pilot test (283 teachers for 57% response rate)

Confirmed changes made after 9-school pilot test.



Design of the sample
Design of the Sample Properties

  • Target of 300 Schools

    • 100 elementary, 100 middle, 100 high schools

    • 150 urban schools, 100 suburban schools, 50 rural schools

    • Of the 150 urban schools, 50 from Wallace grantee districts and 50 from Wallace grantee states

    • 75 schools from each of four geographic areas

  • The Obtained Sample

    • 60 districts (9 Wallace districts)

    • 235 schools for principal data, 253 schools for supervisor data, 245 schools for teachers data

    • 8,863 teachers

    • 218 schools for which there was data from all three respondent groups

    • 50% of the schools had a 68 percent teacher response or better



Performance level descriptors
Performance Level Descriptors Properties

  • Distinguished

    • A distinguished leader exhibits leadership behaviors of core components and key processes at levels of effectiveness that over time are virtually certain to influence teachers to bring the school to a point that results in strong value-added to student achievement and social learning for all students.

  • Proficient

    • A proficient leader exhibits leadership behaviors of core components and key processes at levels of effectiveness that over time are likely to influence teachers to bring the school to a point that results in acceptable value-added to student achievement and social learning for all students.

  • Basic

    • A leader at the basic level of proficiency exhibits leadership behaviors of core components and key processes at levels of effectiveness that over time are likely to influence teachers to bring the school to a point that results in acceptable value-added to student achievement and social learning for some sub-groups of students, but not all.

  • Below basic

    • A leader at the below basic level of proficiency exhibits leadership behaviors of core components and key processes at levels of effectiveness that over time are unlikely to influence teachers to bring the school to a point that results in acceptable value-added to student achievement and social learning for students.


The bookmark procedure
The Bookmark Procedure Properties

Twenty-two panelists: Ten principals, four teachers, four supervisors of principals, two researchers of school leadership, and two education policymakers

Item-ordered booklet based on the National Field Trial (items from Form A)

Panel met August 12 and 13, 2008


The results
The Results Properties

  • The cut between proficient and basic was set at 3.60.

  • The cut between distinguished and proficient was set at 3.77.

  • The cut between basic and below basic was set at 3.42.

  • The impact data are

    • 30% of the principals fall below basic

    • 50% fall below proficient

    • 70% fall below distinguished

  • While the panel was highly positive about the experience and felt comfortable with where they had set the proficient cut, some expressed concern about having set the distinguished cut too low and the below basic cut too high.


Post standard setting reconsideration
Post-Standard Setting Reconsideration Properties

  • In a follow-up email, panelists were given three options to consider

    • Option 1 – Leave the cuts where they are.

    • Option 2 – Move the below basic cut to be consistent with the table that set the lowest value (3.29) and the cut for distinguished consistent with the table that set the highest value (3.87).

    • Option 3 – The same as Option 2 except the distinguished cut is moved to 4.0, requiring that a principal on average get a score of 4 on the 5-point scale.

  • Panelists were given impact data for each possibility across the three options.

  • All 22 panelists responded.

    • 21 favored moving below basic to 3.29

    • 18 favored moving distinguished to 4.00

  • The decision was to go with Option 3, thus the cut scores are 3.29, resulting in 17% of principals below basic, 3.60, resulting in 50% of the principals below proficient, and 4.0 for distinguished, resulting in 14.2% of principals designated as distinguished.


Score reports

Score Reports Properties


Interpretation of rating scale results
Interpretation of Rating Scale Results Properties

Descriptive Analysis

Total Score

Core Components Subscale Scores

Key Process Subscale Scores

Norm-Referenced Profiles

Principal

Teacher

Supervisor

Total respondent composite

Criterion-Referenced Profiles

Distinguished

Proficient

Basic

Below basic







Val ed and professional growth
VAL-ED and Professional Growth Properties

Cell-by-cell feedback highlights up to 6 potential areas of growth.

Behaviors from these 6 domains are listed.

Areas of growth provide principals with information about key targets for professional development.


Leadership behaviors for possible improvement example of a potential area of growth
Leadership Behaviors for Possible Improvement PropertiesExample of a potential area of growth:


ad