What do the student survey questions tell us on the cats assessment
Download
1 / 18

What Do the Student Survey Questions Tell Us - PowerPoint PPT Presentation


  • 725 Views
  • Uploaded on

What Do the Student Survey Questions Tell Us on the CATS Assessment? By Natalie Bolton Stiglitz,PhD Office of Civic Education and Engagement University of Louisville September 23, 2008 School Characteristics (independent variables) Student Demographics (independent variables) Gender

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'What Do the Student Survey Questions Tell Us' - ostinmannual


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
What do the student survey questions tell us on the cats assessment l.jpg

What Do the Student Survey Questions Tell Us on the CATS Assessment?

By Natalie Bolton Stiglitz,PhD

Office of Civic Education and Engagement

University of Louisville

September 23, 2008


Slide2 l.jpg

School Assessment? Characteristics

(independent variables)

Student Demographics

(independent variables)

Gender

Teacher credentials

Ethnicity

KCCT S.S. Student Questionnaire

Student Achievement

in social studies on

Grade 8 2006 KCCT

(dependent variable)

SES

Duration S.S.

Courses

Disability

Excellence and Equity in Social Studies Education


School characteristics 2003 2006 kentucky school report cards l.jpg
School Characteristics Assessment?2003-2006 Kentucky School Report Cards

Teacher Credentials

school percentage of classes taught by teachers certified by subject and grade level

school percentage of classes taught by teachers with a major, minor, or the equivalent in the subject being taught

school percentage of teachers with a Masters degree or greater or the equivalent

school average years of teaching experience


Slide4 l.jpg
School Characteristics Assessment?2006 Kentucky Performance Report S.S. Questionnaire Data and2003-2006 Kentucky LEAD Report

Aggregated student S.S. KCCT questionnaire

Eight items address social studies pedagogy (e.g., emphasis on usage of discussion, textbooks, worksheets, small groups, focus on current events, essential questions, scoring guides, open response questions)

One item addresses the degree to which students feel the assessment is aligned with material covered in class

Two items address students’ motivation for and evaluation of their performance

School social studies average hours instruction 2003-2006, LEAD Report


Student demographics 2006 kentucky performance report s s disaggregated data l.jpg
Student Demographics Assessment?2006 Kentucky Performance Report S.S. Disaggregated Data

Gender

Male= 0 and Female = 1

Ethnicity

White= 0, African-American = 1, Hispanic = 2, Asian = 3, and Other = 4

Socio-economic status (SES)

Free/reduced lunch approved= 0 and

Free/reduced lunch not approved = 1

Disability

No disability= 0 and Disability = 1



What is the greatest predictor on grade 8 s s kcct7 l.jpg
What is the greatest predictor on grade 8 s.s. KCCT? Assessment?

Hierarchical Regression Results (N = 32,833):

Model 1: Gender and ethnicity accounted for 5% of the variance in the total KCCT grade 8 social studies score

Model 2: Disability status explained 10% of the variance in total social studies score over and above gender and ethnicity

Model 3: SES status explained 5% of the variance in the total KCCT grade 8 social studies score over and above gender, ethnicity, and disability

or

Total model explained 19% of the variance in total KCCT grade 8 social studies score

Student Achievement in

social studies


Opportunity to learn otl l.jpg
Opportunity-to-Learn (OTL) Assessment?

“Whether or not…students have the opportunity to study a particular topic or learn how to solve an articulate type of problem presented by the test” (Husen, 1967)

Purpose- maximize fairness and equity for students (Porter, 1993)

Stevens and Grymes (1993)- OTL measures

content coverage- how much of the curriculum is studied

content exposure- the depth to which the curriculum is studied

content emphasis- particular components emphasized in the curriculum

quality of instructional delivery- the way the curriculum is taught

teacher professional characteristics- as the development of content knowledge and knowledge and use of diverse types of assessment

school support systems- affect student learning opportunities such as the amount of time allotted for teacher collaboration and time for teachers to become part of an activeprofessional community



What is the greatest predictor on grade 8 s s kcct10 l.jpg
What is the greatest predictor on grade 8 s.s. KCCT? Assessment?

Hierarchical Regression Results (N = 251):

  • Model 1: Mean school demographics accounted for 21% of the variance in the total KCCT grade 8 school social studies score

  • Model 2: Teacher credentials explained 4% of the variance in total social studies school score over and above mean school demographics

  • Model 3: Mean school student KCCT s.s. questionnaireitems explained 20% of the variance in the total KCCT grade 8 school social studies score over and above mean school demographics and teacher credentials

    or

    Total model explained 44% of the variance in total KCCT grade 8 school social studies score


Slide11 l.jpg

In addition to demographics and teacher credentials, which s.s. student questionnaire item is the greatest predictor of student achievement on grade 8 s.s. KCCT?


Slide12 l.jpg

In addition to demographics and teacher credentials, which s.s. student questionnaire item is the greatest predictor of student achievement on grade 8 s.s. KCCT?

Hierarchical Regression Results (N = 251):

  • Most influential individual predictors on the 2006 grade 8 KCCT included:

    + Student reports of using essential questions

    + Student reports of alignment between coursework and test items

    • Disability status

    • Being African American

    • Using textbooks

    • Discussing current events


Slide13 l.jpg
If you know a school’s student demographics, to what degree can you predict student achievement on grade 8 s.s. KCCT between schools?


Slide14 l.jpg
If you know a school’s student demographics, to what degree can you predict student achievement on grade 8 s.s. KCCT between schools?

Hierarchical Linear Model:

Random effects one-way analysis of variance model results:

  • 13% of the variance in s.s scores was between schools


Slide15 l.jpg
If you know a school’s student demographics, to what degree can you predict student achievement on grade 8 s.s. KCCT within a school?


Slide16 l.jpg
If you know a school’s student demographics, to what degree can you predict student achievement on grade 8 s.s. KCCT within a school?

Hierarchical Linear Model:

  • Random coefficient model results:

    • 23% of the within school variance in s.s. scores were predicted from the four Level 1 predictors (gender, ethnicity, SES, and disability)


Slide17 l.jpg

If you control student and school demographics, which two grade 8 s.s. questionnaire items predict almost 33% of student achievement on grade 8 s.s. KCCT?


Slide18 l.jpg

If you control student and school demographics, which two grade 8 s.s. questionnaire items predict almost 33% of student achievement on grade 8 s.s. KCCT?

HLM findings indicated that two OTL conditions (student reports of alignment between s.s. content taughtand s.s. test items and students’ perceptions of how well they did on the assessment) even after controlling for student and school demographics, significantly predict almost one-third of achievement in s.s. on the grade 8 KCCT.


ad