office for evaluation and needs assessment social science research institute university of hawaii n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Office for Evaluation and Needs Assessment Social Science Research Institute University of Hawaii PowerPoint Presentation
Download Presentation
Office for Evaluation and Needs Assessment Social Science Research Institute University of Hawaii

Loading in 2 Seconds...

play fullscreen
1 / 21

Office for Evaluation and Needs Assessment Social Science Research Institute University of Hawaii - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

Evaluation of Hawaii’s Supplemental Educational Services Program DRAFT Evaluation Plan Presentation to Service Providers and Complex Areas April 2006. Office for Evaluation and Needs Assessment Social Science Research Institute University of Hawaii. Evaluation Team.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Office for Evaluation and Needs Assessment Social Science Research Institute University of Hawaii' - doris-hurley


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
office for evaluation and needs assessment social science research institute university of hawaii

Evaluation of Hawaii’s Supplemental Educational Services ProgramDRAFT Evaluation PlanPresentation to Service Providers and Complex Areas April 2006

Office for Evaluation and Needs Assessment

Social Science Research Institute

University of Hawaii

evaluation team
Evaluation Team
  • Dr. Judith Inazu, P.I., Associate Director, Social Science Research Institute
  • Dr. Shuqiang Zhang, Statistician, College of Education
  • Dr. Daniel Anderson, Consultant, Planning and Evaluation, Inc.
  • Dr. Aiko Oda, Consultant, Planning and Evaluation, Inc.
  • Ms. Nancy Marker, Educational Specialist, Social Science Research Institute
  • Dr. Patty Reiss, Lecturer, College of Education
  • Ms. Julie Holmes, Graduate Assistant, Social Science Research Institute
goal of ses
Goal of SES

To increase the academic achievement of economically disadvantaged students in low-performing schools by providing additional academic instruction outside of the regular school day.

purpose of evaluation
Purpose of Evaluation

To determine the effectiveness of service providers in increasing students’ academic achievement.

Providers must:

► Increase students’ achievement for 2 consecutive years, and

► Provide services consistent with applicable federal, state, and local health, safety, and civil rights requirements.

guiding questions
Guiding Questions

1. Effectiveness

Did the provider increase student achievement in reading and/or mathematics?

  • Customer Satisfaction

Are parents, schools, and complex areas involved in SES satisfied with the service provider?

  • Service Delivery

Did the provider comply with applicable federal and state laws and regulations, and contractual procedures and requirements associated with the delivery of SES?

interim and final evaluation plans
Interim and Final Evaluation Plans

Interim Evaluation Plan

Year 1 (Fall 2005 - Summer 2006)

Year 2 (Fall 2006 - Summer 2007)

Year 3 (Fall 2007 - Summer 2008)

Final Evaluation Plan

Year 4 (Fall 2008 - Summer 2009)

measuring academic achievement
Measuring Academic Achievement

In Year 1 of the Evaluation, academic achievement will be the sole criterion for evaluating service providers

measuring academic achievement1
Measuring Academic Achievement
  • Analysis to be conducted separately for each grade by subject matter, and for each provider.
  • Example: For 4th graders who received tutoring in reading by College Connections.
slide9
Minimum Data Set for AnalysesData on all SES-eligible students. Arrayed by grade level, subject matter, and service provider.
analyses
Analyses

Analyses of Covariance (ANCOVA) with prior HSA score as the covariate, SES status (yes/no) as the binary independent variable, and 2006 HSA score as the dependent variable.

The ANCOVA asks the question, “If you hold the pre-test scores constant (since all students begin at different levels of performance), is there a significant difference in post-test scores between students who received tutoring and students who did not?”

analyses for students without pre test scores
Analyses for Students Without Pre-test Scores

Conduct t-test analyses (difference between mean scores)

  • Compare 2006 HSA test scores between students who received tutoring and those who did not.
  • Again, analyses conducted separately by grade level, subject matter, and for each service provider.
measuring customer satisfaction
Measuring Customer Satisfaction

Pilot tested with selected samples in

Years 1-3:

  • Parents
  • Teachers
  • Complex Area Administrators
  • Principals
  • Students (To-be-determined)
measuring customer satisfaction1
Measuring Customer Satisfaction

Pilot testing to begin in May 2006

  • Paper and pencil questionnaires with a select sample of parents, teachers, complex area administrators, and principals.
  • Translation requirements for parents.
  • Internet survey for teachers, school administrators, and complex areas.
sample questions parental satisfaction
Sample Questions: Parental Satisfaction

1. The school staff was qualified and supportive in helping me get tutoring for my child.

  • Yes
  • No

2. I would recommend this tutor to other parents.

  • Yes
  • No

3. My child's school work improved because of the tutoring.

  • Yes
  • No

4. My child found tutoring was a positive experience.

  • Yes
  • No

5. The tutors were good at reporting to me about my child’s progress.

  • Yes
  • No

6. Overall how would you rate this tutor?

  • Strong
  • Average
  • Weak
measuring service delivery
Measuring Service Delivery

Pilot testing to begin in May 2006

  • Site visits and observations of tutoring sessions
  • Interview(s) with contact person(s) at the site
  • Document review during site visit
  • Self-administered compliance checklist
instructional observation
Instructional Observation
  • Clear academic expectations are set and articulated
  • Instruction is on task without interruption
  • Criterion material is covered with instructor’s direction
  • Instructional time is adequate for material yet flexible
  • Learning modalities are active, variable and appropriate
  • Instructional pacing is appropriate for student’s interest and ability
  • Student progress is frequently assessed
  • Evidence of mutual respect, positive interaction and feedback
  • Evidence of the instructor’s enthusiasm and interest
instructional delivery
Instructional Delivery
  • Student-teacher ratio
  • Computer, lectures, etc.
compliance checklist examples
Compliance Checklist (Examples)

SES Provider Assurances

  • Use research-based strategies designed to improve academic achievement
  • Offer instruction consistent with state academic content and achievement standards
  • Meet federal, state and local health, safety and civil rights law
  • Provide services that are secular, neutral, and non-ideological
  • Provide information on student’s progress, as agreed upon with the LEA
  • Remain financially viable
  • Abide by Family Educational Rights and Privacy Act of 1975
what we ll need from you
What We’ll Need From You

Complex Areas

  • Written permission to be on campus to conduct site observations
  • Names and addresses of parents
  • Completion of satisfaction survey
what we ll need from you1
What We’ll Need From You

Service Providers

  • Assistance in scheduling site visits
  • Set aside time for an interview
  • Documents available for review (e.g., attendance logs, student records, pre/post test reports)
  • Completion of compliance checklist