Download

CalTPA Coordinators’ Conference April 11, 2011 9:00 a.m. — 3:00 p.m. California Baptist University






Advertisement
/ 51 []
Download Presentation
Comments
salena
From:
|  
(3645) |   (0) |   (0)
Views: 121 | Added:
Rate Presentation: 0 0
Description:
CalTPA Coordinators’ Conference April 11, 2011 9:00 a.m. — 3:00 p.m. California Baptist University. Agenda. CalTPA Steering Committee. CalTPA Coordinator Contact Info. Please check the laptop computer on the sign in table and update your program’s CalTPA Coordinator Contact information.
CalTPA Coordinators’ Conference April 11, 2011 9:00 a.m. — 3:00 p.m. California Baptist University

An Image/Link below is provided (as is) to

Download Policy: Content on the Website is provided to you AS IS for your information and personal use only and may not be sold or licensed nor shared on other sites. SlideServe reserves the right to change this policy at anytime. While downloading, If for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.











- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -




Caltpa coordinators conference april 11 2011 9 00 a m 3 00 p m california baptist university l.jpgSlide 1

CalTPA Coordinators’ Conference April 11, 20119:00 a.m. — 3:00 p.m.California Baptist University

Agenda l.jpgSlide 2

Agenda

Caltpa steering committee l.jpgSlide 3

CalTPA Steering Committee

Caltpa coordinator contact info l.jpgSlide 4

CalTPA Coordinator Contact Info

  • Please check the laptop computer on the sign in table and update your program’s CalTPA Coordinator Contact information

Caltpa coordinators conference feedback forms l.jpgSlide 5

CalTPA Coordinators’ Conference Feedback Forms

During the conference, use the conference feedback forms to give us constructive feedback on each of the sessions.

Your comments will be instrumental in fine tuning current policy and determining future topics of discussion and direction for the CalTPA

Please hand in at the end of the day

Since july 2008 we have l.jpgSlide 6

Since July 2008 we have…

  • Built out the SSP task from 5 to 20 different content areas.

  • Developed the online CalTPA Coordinators’ resource site that includes assessor training and recalibration resources. (Sparrow)

  • Posted a TPA / CalTPA section on the CTC website

  • Rewritten the CalTPA Teacher Candidate Handbook

  • Developed the CalTPA Implementation Manual

Since july 2008 continued l.jpgSlide 7

Since July 2008 Continued…

  • Initiated the TPA Users Advisory Council and the CalTPA Steering Committee

  • Developed the annual TPA data collection process

  • Contracted with ETS to provide ongoing psychometric support

  • Continued to provide Assessor and Lead Assessor Training

  • Held a yearly CalTPA Coordinators’ Conference

  • Established TPA and CalTPA Listservs

  • Provided ongoing program support

Caltpa online resources l.jpgSlide 8

CalTPA Online Resources

  • CalTPA section of the CTC website

    • Candidate Handbook

    • Implementation Manual

    • Tasks and Rubrics

    • State Sponsored Assessor Training Calendar

    • CalPTA Listserv Registration

  • CalTPA Coordinators’ Website (Sparrow)

    • Assessor Training Materials

    • State Sponsored Recalibration Tasks, ROEs and Scores

New caltpa bm and isc tasks l.jpgSlide 9

New CalTPA BM and ISC Tasks

New caltpa bm and isc tasks cont l.jpgSlide 10

New CalTPA BM and ISC Tasks Cont...

  • New DI and AL Tasks

    • Validated by the Commission and ETS in October 2010

    • Completed and recently field tested at a State Sponsored Training

  • New SSP and CTE Tasks

    • Validated by the Commission and ETS in March 2011

    • Are still in the final selection and editing process

New caltpa bm and isc tasks cont11 l.jpgSlide 11

New CalTPA BM and ISC Tasks Cont...

  • DI Score Validation Concerns

    • He fits the requirement of being an English learner in a class where there are very few students who do.

    • All of the student permission forms along with student and parent names were included in the response.

    • Included a class picture.

    • I like this student personally and I am trying to use all my resources to help this students succeed. He is smart, but struggles in school due to his language barrier.

New caltpa bm and isc tasks cont12 l.jpgSlide 12

New CalTPA BM and ISC Tasks Cont...

  • AL Score Validation Concerns

    • Student work was not submitted

    • I selected Mario because he struggles academically this year. He also seems to have a speech problem.

    • The student is a very likeable student with an excellent sense of humor. She has diabetes and misses a great deal of school.

    • I selected this student because he has special needs in the classroom. Although he has not yet been tested for any learning disabilities or emotional disorders, he is unable to perform in the classroom without constant adaptations.

New caltpa bm and isc tasks cont13 l.jpgSlide 13

New CalTPA BM and ISC Tasks Cont...

  • CTE Score Validation Concerns

    • Video was not submitted or would not play

    • Student work was not submitted

New caltpa bm and isc tasks cont14 l.jpgSlide 14

New CalTPA BM and ISC Tasks Cont…

  • New CTE videos will be posted on Sparrow where they can viewed either online or downloaded as MP4 files

  • All new initial calibration case should be posted on Sparrow by July 1st, 2011

Tpa and accreditation l.jpgSlide 15

TPA and Accreditation

Terry Janicki, Administrator

tjanicki@ctc.ca.gov

Mike Taylor, Consultant

mtaylor@ctc.ca.gov

Commission on Teacher Credentialing

Tpa and accreditation16 l.jpgSlide 16

TPA and Accreditation

  • Biennial Reports: Assessor data will be submitted

  • Program Assessment: Review process for Standards 17-19

  • Site Visits: Resources are being developed for use at the site visit

Biennial report l.jpgSlide 17

Biennial Report

Changes to the Biennial Report reports due in Fall 2011 from Past Reports

  • The specific information needed is outlined in PSA 10-17 and is related to the assessors used in the implementation of the TPA.

  • PSA10-17 http://www.ctc.ca.gov/educator-prep/PS-alerts/2010/PSA-10-17.pdf.

Changes to biennial report l.jpgSlide 18

Changes to Biennial Report

  • Number of Assessors

  • Assessor Initial Training and Recalibration

  • Data on Reliability Related to Double Scoring

  • Modifications made to assessor selection, training, recalibration.

Changes in the program assessment review of standards 17 19 l.jpgSlide 19

Changes in the Program Assessment Review of Standards 17-19

  • A subset of BIR reviewers with particular expertise in the TPA will review the responses to Standards 17-19

  • Ensure a fair and rigorous process for the review of Standards 17-19 regardless of TPA model

  • Preliminary Findings of Program Assessment reviewers will still be confirmed through interviews and the review of other evidence by BIR members at the site visit

Guiding questions in the program assessment review of standards 17 19 l.jpgSlide 20

Guiding Questions in the Program Assessment Review of Standards 17-19

  • Questions are not intended to replace the TPA related standards, but rather to guide Program Assessment readers to ask critical, but uniform questions of each program’s response

  • Institutions preparing responses may also find these questions helpful as they prepare program assessment documents

Guiding questions in the site review of standards 17 19 l.jpgSlide 21

Guiding Questions in the Site Review of Standards 17-19

  • In PSA10-17 Appendix A are a listing of Considerations for Site Visit for Standards 17-19

  • Site visit team must interview the individual(s) responsible for the implementation of the TPA. Possible/Likely interview subjects for each component of the standards are indicated in Appendix A

Short break l.jpgSlide 22

Short Break

Calibration recalibration l.jpgSlide 23

Calibration & Recalibration

Katie Pedley

Educational Testing Service

Ilene Foster

Claremont Graduate University

Mick Verdi

CSU San Bernardino

Calibration l.jpgSlide 24

Calibration

  • CCSSO & Association of Test Publishers “Operational Best Practices for Statewide Large-Scale Assessment Programs” (2010):

    • “Processes should be established to ensure consistent scoring results of open-ended responses….” (14.2)

    • “Individual and group rater performance should be measured and analyzed regularly” (14.2.2)

  • Calibration is a check that assessors are “on track” prior to scoring

Why calibrate l.jpgSlide 25

Why calibrate?

  • Established link between calibration and scoring accuracy (Ricker-Pedley, 2010)

  • Opportunity to train or refresh on scoring rubric

  • A way to help control rater leniency and severity (Congdon & McQueen, 2000; Lumley & McNamara, 1995).

  • Creating documentation that assessors met performance standard prior to scoring

    • In case of legal challenge of scoring process

Other programs l.jpgSlide 26

Other Programs

  • Praxis, GRE, TOEFL, TOEIC

    • Initial certification (longer set of cases to score)

    • Subsequent calibration prior to EACH time assessors score—assessors who do not meet a scoring standard are dismissed for the day.

    • Significant evidence from these programs that even good, accurate raters can have “off” days.

Recalibration l.jpgSlide 27

Recalibration

GENERAL IDEAS

  • Use Multiple Methods for Recalibration

  • Be Flexible

  • Document How and When Recalibration

    Occurs

    4. Check ROEs for details

Recalibration at csusb l.jpgSlide 28

Recalibration at CSUSB

  • 1. Calibration Every Quarter

    • Randomly select a percentage of each assessors and double score

    • Must have at least 80% agreement and greater than 50% on point

      2. Indicator of Validation

      3. Calibration Meeting

      4. On line recalibration or retesting

      5. Retraining and retesting

Recalibration at cgu l.jpgSlide 29

Recalibration at CGU

  • On-line calibration

  • Mix previously double scored cases in the mix of new assessments. Look for consistency of scoring as well as accuracy filling out ROE with specificity.

  • The ROE is so important for remediation.

  • CGU only assesses a specific Cal TPA once a year (except for resubmissions) and therefore recalibration is needed for everyone at each scoring session.

Recalibration table talk l.jpgSlide 30

Recalibration Table Talk

  • Discuss different ways each is recalibrating at their site.

  • How are you keeping track of recalibration?

  • What type of documentation do you keep?

  • What issues or concerns are arising due to recalibration?

No host lunch in the cafeteria l.jpgSlide 31

No - host Lunch in the Cafeteria

Feedback informing successful tpa candidates michael cosenza nedra crow l.jpgSlide 32

Feedback: Informing Successful TPA CandidatesMichael Cosenza & Nedra Crow

Purposes l.jpgSlide 33

Purposes

  • Provide examples of candidate feedback forms.

  • Answer questions.

  • Discuss participants’ forms.

Ctc candidate feedback l.jpgSlide 34

CTC & Candidate Feedback

Required:

  • Overall holistic score.

    Suggested:

  • Rubric that accompanies the score.

    Not permitted:

  • Giving the candidate the ROE.

  • Even though the ROE may not be shared with the candidates, additional feedback may be provided.

Candidate feedback specifics l.jpgSlide 35

Candidate Feedback: Specifics

National:

  • Provides feedback (passing & non-passing) using:

    • TPA’s rubric language.

    • Specific Task’s sections.

  • Feedback transmitted through TaskStream.

    Cal Lutheran:

  • Provides detailed feedback for remediation but not for those who pass.

  • Just beginning to provide comments via a local rubric in TaskStream.

National university l.jpgSlide 36

National University

Task-Performance Assessment Diagnostic

TPAD

Go!

California lutheran university l.jpgSlide 37

California Lutheran University

  • Local Rubric

Discussion l.jpgSlide 38

Discussion

  • Is your program providing candidate feedback in addition to their score?

  • If yes, how does your program provide this feedback?

  • Are you using some type of local rubric?

Remediation l.jpgSlide 39

Remediation

Keith Walters

Cal Baptist University

Sara Cloutier

Consultant

Stacy Schmidt

CSU Bakersfield

Remediation through independent study l.jpgSlide 40

Remediation through Independent Study

  • Candidates register for a 1 unit TPA class for SSP, DI, and AL task

  • Candidates register for a 2 unit TPA class for the CTE task

  • Task is due the 7th week of the quarter

  • Score of 1 or 2 is not passing and requires remediation

Remediation through independent study41 l.jpgSlide 41

Remediation through Independent Study

  • TPA coordinator is available for all candidates

  • Reviews areas of weaknesses with candidates in a one-on-one setting

  • Required to revise and resubmit in 10 days

  • Score of 1 or 2 is not passing

  • Candidates are not allowed to progress in program

  • Candidates sent for remediation

Remediation through independent study42 l.jpgSlide 42

Remediation through Independent Study

  • Extended University Course

  • 1-2-3 unit courses

  • Less expensive

  • One-on-one remediation

  • Two more opportunities to pass

Remediation through independent study43 l.jpgSlide 43

Remediationthrough Independent Study

  • Candidates make schedule with TPA coordinator

  • 10 hours of one-on-one help available per unit

  • Candidate initiates meetings

Slide44 l.jpgSlide 44

Teach to the test

NOT

to the score

Foundational abilities ppt presentations l.jpgSlide 45

Foundational Abilities – PPT Presentations

  • Using Content Specific Pedagogy

    • TPE #1, and #9

  • Learn About Students

    • TPE #3, #6, #7, and #8

  • Creating a Positive Classroom Environment

    • TPE #2, #5, #10, and #11

  • Establishing Learning Goals

    • TPE #1 and #8

  • Plan Instruction and Assessments

    • TPE #3, #4, #6, #8, and #9

  • Make Adaptations

    • TPE #5, #6, #7 and #8

  • Analyzing Student Learning and Lesson Effectiveness

    • TPE #2, #3, and #5

  • Professional Development and Reflection

    • TPE #13

  • A nalyzing l.jpgSlide 46

    Analyzing

    • Learn About Students (#2)

    • Making Adaptations (#6)

    • Planning Instruction (#5)

    • Content Specific Pedagogy (#1)

    • Analyzing Student Learning (#7)

    B reak down l.jpgSlide 47

    Break Down

    Connection to prior information

    Type of question

    Specific TPE focus – go to PPT for help

    Positive Environment (#3)

    Analyzing Learning (#7)

    C onfirm l.jpgSlide 48

    Confirm

    Address all portions of prompt

    Concept embedded in prompt

    Connected clearly and consistently

    Tpa data collection l.jpgSlide 49

    TPA Data Collection

    Mike Taylor

    CTC Consultant

    Tpa data collection50 l.jpgSlide 50

    TPA Data Collection

    • 2011 data collection will be for the 09-10 academic year

    • Similar process to last year

      • Excel templates via email

    • A few changes

      • No SSNs

      • No encrypted email

      • Templates pre-populated with candidate names

      • No candidate scores, only pass/fail information

    • Templates distributed at the beginning of May, due back to CTC end of July

    Caltpa questions answers caltpa steering committee l.jpgSlide 51

    CalTPAQuestions & AnswersCalTPA Steering Committee


    Copyright © 2014 SlideServe. All rights reserved | Powered By DigitalOfficePro