slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
A Dynamic Model for Gifted Program Improvement PowerPoint Presentation
Download Presentation
A Dynamic Model for Gifted Program Improvement

Loading in 2 Seconds...

play fullscreen
1 / 26

A Dynamic Model for Gifted Program Improvement - PowerPoint PPT Presentation


  • 245 Views
  • Uploaded on

Designing and Utilizing Evaluation for Gifted Program Improvement Elissa F. Brown, Ph.D. Director, Center for Gifted Education College of William and Mary efbrow@wm.edu (757) 221-2362 S.C. Consortium for Gifted Education Charleston, SC November 29, 2006.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'A Dynamic Model for Gifted Program Improvement' - badu


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Designing and Utilizing Evaluation for Gifted Program ImprovementElissa F. Brown, Ph.D.Director,Center for Gifted EducationCollege of William and Maryefbrow@wm.edu(757) 221-2362S.C. Consortium for Gifted EducationCharleston, SCNovember 29, 2006

a dynamic model for gifted program improvement
A Dynamic Model for Gifted Program Improvement

PLAN

(Program / Curriculum)

DO

(Implementation /

Instruction)

ACT

(Plan of Action /

Improvement)

STUDY

(Evaluation /

Assessment)

Center for Gifted Education – School of Education – The College of William and Mary

key beliefs
Key Beliefs
  • The fundamental role of evaluation is to provide information that can be used to improve and advance the state of the art of gifted programs.
  • Evaluation research is a collaborative process among stakeholders including the state legislature, the state department of education, local school districts, and the contractor.
  • The use of multiple data sources helps to illuminate the complexity and salience of issues needing to be considered.
planning a program evaluation identifying stakeholders
Guiding Questions

Who has a vested interest in the gifted program?

Who needs the information for program improvement?

Are there others that may be impacted by the program evaluation?

Matching stakeholder groups

Parents, teachers, students, administrators

Teachers, administrators, school board

Support staff, Dept. of Education, regular education teachers

Planning a program evaluation: Identifying Stakeholders
evaluation questions
Evaluation Questions

Question 1: To what extent are the stated mission and goals of the gifted program fulfilled in their actual operation?

Question 2: To what extent is the gifted program meeting the needs of identified students as perceived by relevant groups?

Question 3: What evidence exists to document positive student performance trends for students participating in the gifted program?

slide6

Evaluation Questions (cont.)

Question 4: What are the program strengths and weaknesses in relation to the state of the art or best practices in gifted education?

Question 5: What are recommendations for program improvement or revision?

data sources
Data Sources
  • Survey of students, parents, teachers, and administrators
  • Focus groups of students, parents, teachers, administrators
  • Interviews with key administrators
  • Classroom observations
  • Student impact data
  • Review of written documents
process of constructing stakeholder surveys in gifted program evaluation
Process of Constructing Stakeholder Surveys in Gifted Program Evaluation
  • Identify stakeholder groups
  • Develop questions for cross-group validation
  • Develop reliable and valid stakeholder surveys
process of constructing stakeholder survey
Process of Constructing Stakeholder Survey
  • Write brief questions
  • Write understandable questions
  • Ask relevant people relevant questions
  • Write uni-dimensional questions
  • Write uni-dimensional categories
  • Write mutually exclusive categories
  • Avoid writing loaded questions
content of questions
Content of Questions
  • Align with different components of NAGC standards
  • Align with program goals and objectives
  • Derive information from preliminary visits
  • Review the literature
nagc program standards
NAGC Program Standards
  • Program design
  • Student identification
  • Curriculum and instruction
  • Socio-emotional guidance and counseling
  • Professional development
  • Program administration and management
  • Program evaluation
focus groups
Focus Groups

Definition:

  • “an interview style designed for small groups” (Berg, 1998)
  • “explicit group interaction to produce data and insights that would be less accessible without the interaction found in the group.” (Morgan, 1988)

Rationale for use in gifted evaluations:

  • To deepen understanding of key gifted program issues, using a multiple perspectives lens
focus groups literature review
Focus Groups:Literature Review
  • Uses in gifted program evaluation have been predominately for confirmatory and triangulation purposes (Fern,2001; Knodel, 1993; Morgan, 1988; Wolff, Knodel, & Sittitrai, 1993).
  • The number of focus groups needed in an evaluation depends on the purpose, issues, and group characteristics (Fern, 2001; Knodel, 1993).
  • Groups should have within group homogeneity and cross-group heterogeneity (Morgan, 1998; Morgan & Kruger, 1993; Stewart & Shamdasani, 1990).
process for developing focus group protocol
Process for Developing Focus Group Protocol
  • Size of group ranges from 6-12 participants.
  • Design questions in a way to allow for cross group comparison in the analysis process
  • Tailor questions for different stakeholder groups
  • Make questions relevant to stakeholders
  • Use open-ended probes
process for administering focus groups
Process for Administering Focus Groups
  • Moderator describes the purpose of the focus group
    • to gain an overall picture of the gifted program from different perspectives
    • to provide an opportunity for small group sharing of participants’ experiences and ideas
  • Moderator reads each question one at a time and elicits responses from participants
    • each participant records their responses on an index card
    • participants talk about their responses
    • a note taker records key points on a flip chart
  • Moderator debriefs session
an example of triangulation of focus group data
An Example of Triangulation of Focus Group Data

*Stakeholder codes: S=Student; RT=Regular Classroom Teacher; A=Administrator; GT=Teachers of Gifted Students; P=Parents

**In a focus group design with five stakeholder groups, the consensus of at least three groups is needed for extracting a theme, following a minimum of 50% consensus rule.

assessing classroom practice purposes
Assessing Classroom Practice:Purposes
  • Conduct classroom observations in multiple instructional contexts.
  • Examine differences in instructional behaviors in different organizational patterns, different teacher groups, and different subject areas.
  • Shape an emerging consensus around acceptable levels of practice in gifted education programs.
assessing classroom practice literature review
Assessing Classroom Practice:Literature Review
  • No documented differentiation practices for gifted in heterogeneous classrooms (84%); (Westberg, Archambault, Dobyns, & Salvin,1993)
  • Ineffective teachers over 3 years resulting in depressed effects on student achievement in math regardless of ability level (Sanders & Rivers, 1996)
  • Positive effects of employing key practices (e.g. critical thinking or metacognition) on student learning in math & science for elementary and middle school levels (Wenglinsky, 2000)
assessing classroom practice literature review20
Assessing Classroom Practice:Literature Review
  • Higher-level reform behavior takes a minimum of two years of intensive training to demonstrate results (Borko, 1993)
  • Content-based curriculum intervention for gifted coupled with staff development results in significant & important growth gains (Little, Feng, VanTassel-Baska, Rogers, & Avery, 2003; VanTassel-Baska, Zuo, Avery & Little, 2002; VanTassel-Baska, Bass, Ries, Poland & Avery, 1998)
assessing classroom practice instrument construction cos r
Assessing Classroom Practice:Instrument Construction (COS-R)
  • Categories are consonant with research on effective teaching practices, teacher reform literature, and teaching high-ability learners
      • Curriculum Planning and Delivery
      • Accommodations for Individual Differences
      • Problem Solving
      • Critical Thinking Strategies
      • Creative Thinking Strategies
      • Research Strategies
assessing classroom practice discussion
Assessing Classroom Practice:Discussion

Teachers of the Gifted:

  • Strong in general teaching strategies (e.g. lesson planning and clarity in directions) but fewer differentiation practices employed than desired
  • Little emphasis placed on accelerative practices
  • Problem-solving behaviors were not routinely observed
  • “Compare/contrast” activities most prevalent critical thinking approach employed
  • Little emphasis placed on metacognition or extension
sample findings across data sources outcome and process of triangulation
Sample Findings Across Data Sources:Outcome and Process of Triangulation

*DR=Document Review, I=Interview, FG=Focus Group, CO=Classroom Observation, S=Survey data

dissemination and utilization map
Dissemination and Utilization Map

-VanTassel-Baska, 2003

slide26
Evaluation has moved from being primarily a reservoir of methods for evaluation to now also becoming and being a reservoir for knowledge about generic patterns of program effectiveness.

-- Patton