Cluster 5 spring 2005 assessment results
This presentation is the property of its rightful owner.
Sponsored Links
1 / 45

Cluster 5 Spring 2005 Assessment Results PowerPoint PPT Presentation


  • 50 Views
  • Uploaded on
  • Presentation posted in: General

Cluster 5 Spring 2005 Assessment Results. Wellness Domain. Cluster 5 Course Requirement. Complete 1 course in each domain. Assessment Day Data Collection Scheme: Pre/Post.

Download Presentation

Cluster 5 Spring 2005 Assessment Results

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cluster 5 spring 2005 assessment results

Cluster 5 Spring 2005 Assessment Results

Wellness Domain


Cluster 5 course requirement

Cluster 5 Course Requirement

  • Complete 1 course in each domain


Assessment day data collection scheme pre post

Assessment Day Data Collection Scheme: Pre/Post

Students in each cohort are tested twice on the same instrument – once as incoming freshmen (0 credit hours) and again in after they’ve completed 45-70 credit hours.

COHORT 1

COHORT 2

Cohort 2: First time for Cluster 5 to have pre/post data since new objectives/instruments.

COHORT 3


Wellness domain instruments

Wellness Domain Instruments

  • Knowledge of Wellness and Health Test (KWH1)

    • Scenario-based multiple-choice test created by the faculty

  • Health and Wellness Questionnaire (HWQ)

    • Part 1: self-report survey to measure student’s own health and wellness behaviors

    • Part 2: 33 health and wellness statements; students identify whether they are facts or myths

  • Well-Being Scale (WBS)

    • Self-report measure of psychological well-being


Results of kwh1

Results of KWH1

  • First: Examine the results of the KWH1 for 644 2nd semester sophomores in Spring 2005

    • Overall Results

    • By JMU vs. Transfer/AP

    • By domain “completers” vs. “non-completers”

    • By which course completed

    • Rel. between course grade and KWH1 score

  • Second: Examine pre/post results for the 394 students who took the KWH1 in both Fall 2003 and Spring 2005

    • Overall pre/post difference

    • Pre/post difference by domain “completers” vs. “non-completers”

    • Pre/post difference by which course completed

    • Pre/post difference by item for “completers”


Overall kwh1 results 2 nd semester sophomores n 644

Overall KWH1 Results:2nd Semester Sophomores(N=644)

On average, students are answering 60% of the items correctly. Most scores are between 19 & 25.


Jmu n 628 vs transfer ap students n 16

JMU (n=628) vs. Transfer/AP Students (n=16)

  • Are students who obtained transfer or AP credit for their courses scoring differently than students who completed their courses at JMU?

    • No.

  • JMU students: M = 21.16, SD = 3.30

  • AP/transfer students: M = 20.38, SD = 3.31

  • Difference was not statistically, t(560)=.93, p = .35, nor practically significant (d = .23).


Domain completers vs non completers

Domain “Completers” vs. “Non-Completers”

“Non-Completers”

N = 82

“Completers”

N = 555

More courses completed, the higher the mean. Differences among the means, however, are of small practical significance (d=.06)

Means for students completing two Wellness classes not reported due to small N


Kwh1 means by course completed completers only

KWH1 Means by Course Completed“Completers” Only

There were minor differences among the KWH1 means (d= .20)


Relationship between kwh1 and course grade

Relationship between KWH1 and Course Grade

There was a positive and moderate relationship between KWH1 Total score and course grade (r = .31) in both GKIN100 and GHTH100.


Pre post results n 394

Pre/Post Results (N = 394)

  • Spring 2005: M = 21.13, SD = 3.24

  • Fall 2003: M = 19.00, SD = 3.07

  • The difference between these means is of medium to large practical significance (d = .67)

Average increase of 6.1% in wellness domain knowledge


Pre post for non completers n 16 vs completers n 355

Pre/Post for “Non-Completers” (n = 16) vs. “Completers” (n = 355)

“Non-Completers”

N = 16

“Completers”

N = 355

Means on the KWH1 for “completers” and “non-completers” are very similar. This indicates that gains made on the KWH1 may simply be a function of maturation, not a function of having completed one’s wellness domain requirement.


Pre post by wellness course completed

Pre/Post by Wellness Course Completed

N = 215

N = 135

The same gains (~ 2 pt increase) can be expected in average KWH1 scores for students fulfilling their wellness domain requirement by taking either GHTH100 or GKIN100.


Pre post difference in items for completers

Pre/Post Difference in Items for “Completers”

Even in Spring 2005, there were still 14 items where less than half of the students were obtaining the correct answer.


Items with large pre post gains

Items with Large Pre/Post Gains

7% or more students answered these items correctly in Spring 2005 than in Fall 2003

There were three items (5,11, & 15) with sizable gains, meaning that ~20% or more of the students responded correctly to these items in Spring 2005 than in Fall 2003.


Items with small or no pre post gains

Items with Small (or No) Pre/Post Gains

Items with no change, or change in the wrong direction are shown below for items that are of medium to extreme difficulty.

Items as a whole don’t seem to be related to any one goal or objective, but they do seem to be related to the recall of specific information.


Conclusions kwh1 2 nd semester sophomore data

Conclusions KWH12nd Semester Sophomore Data

  • 2nd semester sophomores are scoring on average ~60%

  • AP/Transfer students are scoring no differently than JMU students

  • “Non-Completers” of the wellness domain requirements are scoring no differently than “Completers”

  • Same average score regardless of what course student took to completed wellness domain requirement

  • There is a relationship between course grade and score, indicating that test content and course content are related


Conclusions kwh1 pre post data 1

Conclusions KWH1Pre/Post Data(1)

  • Average increase of 6.1% in wellness domain knowledge between 0 credit hours and 45-70 credit hours

    - 53.9%  60%

  • “Completers” and “Non-Completers” both make similar gains over time

    • Might be evidence that score gains are a result of maturation not a result of taking wellness course

    • the lack of a sizable gain for the “completers” in comparison to the “non-completers” may be a function of the instrument used to assess the wellness domain goals and objectives.

  • Same gains (~ 2 pt increase) are being made on the test for students who fulfilled their requirement by taking different wellness courses


Conclusions kwh1 pre post data 2

Conclusions KWH1Pre/Post Data(2)

  • Still 14 items in Spring 2005 where < 50% of the students were obtaining the correct answer

  • Not seeing gains on easy items

  • Large increases were seen for items in the middle difficulty range as well as several items in the extreme difficulty range

  • Three items (5, 11, & 15) had sizable gains

    - ~ 20% or more of students responded correctly in Spring 2005 than in Fall 2003

  • Items with no change, or change in the wrong direction were not related to any one goal or objective.

    - Seem to be related to the recall of specific information

    - Lack of improvement may be a function of the item being confusing or ambiguous and in need of revision


Results of hwq1 part1

Results of HWQ1- Part1

  • First:Examine the results of the HWQ1-Part1 for 644 2nd semester sophomores in Spring 2005

    • Overall Results

    • By JMU vs. Transfer/AP

    • By domain “completers” vs. “non-completers”

    • By which course completed

    • Rel. between course grade and KWH1 score

  • Second:Examine pre/post results for the 305 students who took the HWQ1-Part1 in both Fall 2003 and Spring 2005

    • Overall pre/post difference

    • Pre/post difference by domain “completers” vs. “non-completers”

    • Pre/post difference by which course completed

    • Pre/post difference by item for “completers”


Overall hwq1 part1 results 2 nd semester sophomores n 644

Overall HWQ1-Part1 Results: 2nd Semester Sophomores(N = 644)

JMU (n = 628) vs.Transfer/AP Students (n = 16)

  • Are students who obtained transfer or AP credit for their courses scoring differently than students who completed their courses at JMU?

    • No.

  • JMU students: M = 40.35, SD = 5.57

  • AP/transfer students: M = 42.06, SD = 5.32

  • Difference was not statistically, t(560)=-1.22, p = .23, nor practically significant (d = .31).


Domain completers vs non completers1

Domain “Completers” vs. “Non-Completers”

“Completers”

N = 555

“Non-Completer”

N = 82

More courses completed, the higher the mean. Differences among the means, however, are of small practical significance (d=.17)


Hwq1 part1 means by course completed completers only

HWQ1-Part1 Means by Course Completed“Completers” Only

There was a minor difference between the HWQ1-Part1 means (d = .14)


Relationship between hwq1 part1 and course grade

Relationship between HWQ1-Part1 and Course Grade

There was a positive yet small relationship between HWQ1 – Part1 Total score and course grade in both GHTH100 (r = .14) and GKIN100 (r = .05).


Pre post results n 305

Pre/Post Results (N = 305)

Average increase of 2.1% in healthy behaviors

  • Spring 2005: M = 40.26, SD = 5.46

  • Fall 2003: M = 38.99, SD = 5.30

  • The difference between these means is of small practical significance (d = .24)


Pre post for non completers n 21 vs completers n 283

Pre/Post for “Non-Completers” (n = 21) vs. “Completers” (n = 283)

“Non-Completers”

N = 21

“Completers”

N = 283

Means for “completers” and “non-completers” are very similar. This indicates that gains in healthy behavior may simply be a function of maturation, not a function of having completed one’s wellness domain requirement.


Pre post by wellness course completed1

Pre/Post by Wellness Course Completed

Did not report the averages for GEIC101 since only 6 students completed that course

N = 165

N = 114

The same gains (~ 1 pt increase) can be expected in average HWQ1- Part1 scores for students fulfilling their wellness domain requirement by taking either GHTH100 or GKIN100.


Pre post difference in items for completers1

Pre/Post Difference in Items for “Completers”

The pattern of averages across items for both administrations is quite similar. Relative frequencies stay about the the same over time.

  • Students reported engaging in Q11 and Q12 either Often or Frequently (4-6 times or 7 or more times/week).

  • There are increases over time in reported frequency of engaging in most healthy behaviors, with the exception of Q9, Q11 and Q12, which are decreasing over time.


Pre post item differences

Pre/Post Item Differences

As second semester sophomores, students are spending less time socializing with friends or family and eating more hamburgers, hotdogs and sausages than they did as incoming freshmen.

The largest increases in healthy behaviors include the frequency with which students exercise to strengthen muscles (Q10) and do stretching exercises (Q13).


Conclusions hwq1 part1 2 nd semester sophomore data

Conclusions HWQ1-Part12nd Semester Sophomore Data

  • AP/Transfer students are scoring no differently than JMU students

  • “Non-Completers” of the wellness domain requirements are scoring no differently than “Completers”

  • Same average score regardless of what course student took to completed wellness domain requirement

  • There is a small, yet positive relationship between course grade and score, indicating that test content and course content are related


Conclusions hwq1 part1 pre post data 1

Conclusions HWQ1-Part1Pre/Post Data(1)

  • Average increase of 2.1% in healthy behaviors between 0 credit hours and 45-70 credit hours

  • “Completers” and “Non-Completers” both make similar gains over time

    • Might be evidence that score gains are a result of maturation not a result of taking wellness course

    • the lack of a sizable gain for the “completers” in comparison to the “non-completers” may be a function of the instrument used to assess the wellness domain goals and objectives.

  • Same gains (~ 1 pt increase) are being made on the test for students who fulfilled their requirement by taking different wellness courses


Conclusions hwq1 part1 pre post data 2

Conclusions HWQ1-Part1Pre/Post Data(2)

  • Same pattern of averages across items for both administrations

  • Almost all healthy behaviors show an increase in how often students are engaging in them

    - Q9, Q11, and Q12 are exceptions, these decrease

  • Students have increased the frequency for which they exercise to strengthen muscles and stretch


Results

Results

  • First: Examine the results of the HWQ1-Part2 for 644 2nd semester sophomores in the Spring 2005

    • Overall Results

    • By JMU vs. Transfer/AP

    • By domain “completers” vs. “non-completers”

    • By which course completed

    • Rel. between course grade and KWH1 score

  • Second: Examine pre/post results for the 305 students who took the HWQ1-Part2 in both Fall 2003 and Spring 2005

    • Overall pre/post difference

    • Pre/post difference by item for “completers”


Overall hwq1 part2 results 2 nd semester sophomores n 644

Overall HWQ1-Part2 Results: 2nd Semester Sophomores (N = 644)

On average, students are answering 60% of the items correctly. The majority of students have scores that range from 22-28.


Jmu n 628 vs transfer ap students n 161

JMU (n = 628) vs.Transfer/AP Students (n = 16)

  • Are students who obtained transfer or AP credit for their courses scoring differently than students who completed their courses at JMU?

    • No.

  • JMU students: M = 24.28, SD = 3.40

  • AP/transfer students: M = 24.31, SD = 3.88

  • Difference was not statistically, t(560)=.04, p = .97, nor practically significant (d = .009).


Domain completers vs non completers2

Domain “Completers” vs. “Non-Completers”

“Completers”

N = 555

“Non-Completers”

N = 82

Although the Part 2 average was higher for “completers”, the difference between the two averages was minor (d = .11).


Hwq1 part2 means by course completed completers only

HWQ1-Part2 Means by Course Completed“Completers” Only

There was no difference in the averages for students taking GHTH100 as opposed to GKIN100 (d = .18)


Relationship between hwq1 part2 and course grade

Relationship between HWQ1-Part2 and Course Grade

r = .25

Removal of the outlier did not substantially change the descriptive statistics and correlations

r = .08


Pre post results n 3051

Pre/Post Results (N = 305)

  • Spring 2005: M = 24.35, SD = 3.33

  • Fall 2003: M = 22.57, SD = 3.39

  • The difference between these means is of moderate practical significance (d = .54)

Average increase of 3.4% in ability to discern “myths” from “facts”


Pre post difference in items for completers2

Pre/Post Difference in Items for “Completers”

  • Statements that were easy for students to identify correctly as myths or facts as freshman (90-100% obtained the correct answer) remain easy for them as second semester sophomores.

  • For the majority of the remaining items, there is improvement over time in students’ ability to identify statements correctly as myths or facts.


Easy items

Easy Items

Items in which 88% or more of incoming freshmen are identifying correctly as myths or facts

Since so many students know these concepts coming into JMU, little to no time needs to be allocated to this content in the wellness domain course


Items with large moderate gains

Items With Large & Moderate Gains


Items with no or very small gains

Items with No or Very Small Gains

Proportion Correct ItemItem Content Fall 03 Spring 05 Diff.


Conclusions hwq1 part2 2nd semester sophomore data

Conclusions HWQ1-Part22nd Semester Sophomore Data

  • 2nd semester sophomores are scoring on average 60%

  • AP/Transfer students are scoring no differently than JMU students

  • “Non-Completers” of the wellness domain requirements are scoring no differently than “Completers”

  • Same average score regardless of what course student took to complete wellness domain requirement

  • There is a small, yet positive relationship between course grade and score, indicating that test content and course content are related


Conclusions hwq1 part2 pre post data

Conclusions HWQ1-Part2Pre/Post Data

  • Average increase of 3.4% in ability to discern between “myths” from “facts” between 0 credit hours and 45-70 credit hours

    - 70%  73.4%

  • Statements easy for freshman (90-100% answered correctly) remained easy for them as 2nd semester sophomores

    - Since so many students know these concepts coming into JMU, little to no time needs to be allocated to this content

  • For the majority of the remaining items, there is improvement over time in students’ ability to identify statements correctly

    - May be a result of them having completed their wellness domain requirement, or maturation

  • More time may need to be allocated towards items with small or no change or statements should be revised


  • Login