Core assessment attributes of a successful program at james madison university
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

Core Assessment: Attributes of a Successful Program at James Madison University PowerPoint PPT Presentation


  • 83 Views
  • Uploaded on
  • Presentation posted in: General

Core Assessment: Attributes of a Successful Program at James Madison University. Dr. Dena Pastor Center for Assessment and Research Studies. TAIR Professional Development Workshop September 2004. James Madison University (JMU). Location: Harrisonburg, VA Students:

Download Presentation

Core Assessment: Attributes of a Successful Program at James Madison University

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Core assessment attributes of a successful program at james madison university

Core Assessment: Attributes of a Successful Program at James Madison University

Dr. Dena Pastor

Center for Assessment and Research Studies

TAIR

Professional Development Workshop

September 2004


James madison university jmu

James Madison University (JMU)

  • Location: Harrisonburg, VA

  • Students:

    • ~15,000 undergraduate

    • ~700 graduate

  • State-funded university operated by its own board of visitors


Jmu infrastructure for assessment

JMU Infrastructure for Assessment

  • Center for Assessment and Research Studies (CARS)

    • Supports JMU assessment endeavors

      • General Education

      • Programs/Majors

      • Student Affairs

      • Alumni

  • 9full-time faculty members & 3 administrative staff

    • Faculty split their time evenly between assessment duties & teaching/research

    • Expertise in IRT, SEM, Validity, Computer-Based Testing, HLM, Examinee Motivation, Performance Assessment, Student Development

  • Offers both M.A. and Ph.D. programs in Assessment & Measurement

    • 10-15 students


Jmu infrastructure for general education

JMU Infrastructure for General Education

  • Dean of General Education

  • Cluster Coordinators

    • Faculty given release time for this task

  • Steering Committee

    • Broad faculty representation

  • Positions allocated to departments to teach General Education and major courses

  • Each cluster undergoes a review process every 5 years


General education at jmu

General Education

General Education at JMU

  • 41 credit hours of courses organized into 5 clusters

Cluster 1:9 credits

Skills for the 21st Century

Critical Thinking, Written & Oral Communication, Information Literacy

Cluster 2: 9 credits

Arts & Humanities

Culture, Philosophy, Fine Arts & Literature.

Cluster 3:10 credits

The Natural World

Quantitative & Scientific Reasoning

Cluster 4: 7 credits

Social & Cultural Processes

Global & American: History, Govt., Economics, Anthropology

Cluster 5: 6 credits

Individuals in the Human Community

Wellness, Psychology & Sociology


General education course offerings

Wellness: Choose one

Sociocultural: Choose one

General Education Course Offerings

Example from Cluster 5’s Wellness & Sociocultural Domains


Core assessment attributes of a successful program at james madison university

Use

Information

Assessment Steps

Maintain

Information

Analyze Information

Collect Information

Select/Design Instruments

Establish

Objectives


Cluster learning objectives

Cluster Learning Objectives

  • Created by cluster faculty with assistance from CARS

  • Start by asking:

    • “What should a student know and be able to do as a result of completing their coursework in this cluster?”

    • What the student knows & is able to do should NOT be tied to specific courses taken within the cluster.


Example of cluster learning objectives

Example of Cluster Learning Objectives

Example from Cluster 5’s Wellness & Sociocultural Domains

After completing Cluster Five students should be able to:

  • Understand the dimensions of wellness, the various factors affecting each dimension, and how dimensions are interrelated.

  • Recognize an individual's level of health and wellness and understand how these levels impact quality of life.

  • Discriminate between ethical and non-ethical practices in the social/behavioral sciences.


Match learning goals with coursework

Match Learning Goals With Coursework

  • Have instructors rate the extent to which the material necessary to achieve each of the goals is emphasized in their course

  • This will help ensure that students are able to achieve goals regardless of which combination of courses they take

A student taking GHTH100 & GPSYC101 should have the same chance of achieving goals as student who takes GKIN100 & GSOCI240


Assessment of learning objectives

Assessment of Learning Objectives

  • Faculty within each cluster decide how to assess the extent to which students are mastering the cluster’s learning objectives

  • Most clusters create a multiple-choice test from scratch, with continual improvements made on the instrument


Assessment tools

Assessment Tools

Almost all assessment tools are

multiple-choice tests created by JMU faculty.


Matching assessment tools to learning objectives

Matching Assessment Tools to Learning Objectives

Example from Cluster 5’s Sociocultural Domain Assessment


Administration of assessment tools

Administration of Assessment Tools

  • Two institution-wide Assessment Days

    • Fall (August): Incoming freshmen tested at orientation

    • Spring (February): Students with 45-70 credits (sophomore/junior) tested

  • All undergraduate classes are cancelled on this day

  • All students are required to participate

    • course registration blocked for those who do not participate

  • Students are assigned to testing rooms by last 4-digits of their id #s where a particular series of instruments are administered


Assessment day data collection scheme repeated measures

Assessment Day Data Collection Scheme:Repeated Measures

COHORT 1

Students in each cohort are tested twice on the same instrument – once as incoming freshmen and again in the second semester of the sophomore year.

COHORT 2

COHORT 3


With this data we can

Compare pre-test scores to post-test scores (same students) of Cluster “Completers” .

Compare Incoming Freshmen with Sophomore Students (different students).

Examine the Effect of Number of Courses.

Correlate Scores and Course Grades.

With this data, we can…

Students completing their cluster courses should score higher on the post-test as sophomores than they did on the pre-test as incoming freshmen.

Sophomores should outperform incoming freshmen.

Students who have completed a larger # of courses within a cluster should outperform students who have completed a smaller # of courses.

There should be a positive correlation between test scores and the grades received by students in the cluster courses.


1 compare pre test scores to post test scores same students of cluster completers

1. Compare pre-test scores to post-test scores (same students) of Cluster “Completers” .

Example from Cluster 4’s American Experience Test


2 compare incoming freshmen with sophomore students different students

2. Compare Incoming Freshmen with Sophomore Students (different students).

Example from Cluster 3’s Natural World Assessment


3 examine the effect of of courses

3. Examine the Effect of # of Courses.

Example from Cluster 3’s Natural World Assessment


4 correlate scores and course grades

4. Correlate Scores and Course Grades.

Example from Cluster 4’s Global Experience Test


Assessment information not collected on assessment day

Assessment Information Not Collected on Assessment Day

These are both Computer-Based Tests administered throughout the school-year at a computer lab on campus. All students must exceed the passing score set by the faculty within their first year.

A random sample of ~400 students taking the Gen. Ed. writing course are asked to submit a portfolio of their best work which is rated according to a 4 point rubric by the faculty.

The GCOM test is a Computer-Based Test embedded within a course: This test serves as the final exam in all sections of the Gen. Ed. communication course.


Other assessment information previously collected on assessment day

Other Assessment Information Previously Collected on Assessment Day


How does general education assessment benefit jmu faculty

How does General Education Assessment Benefit JMU Faculty?

  • Collaboration among faculty in different disciplines

  • Opportunity to communicate with one another

  • Feel ownership over assessment tools, more invested in the results

  • Data may help support resources they would like to acquire

  • Gain skills that they can take back into the classroom

  • Opportunities for scholarship


How does general education assessment benefit jmu students

How does General Education Assessment Benefit JMU Students?

  • Receive instruction that is more tailored to their needs

  • Receive instruction that is more consistent across sections

  • Held accountable for knowledge/skills (ISST, Tech Level tests, GCOM final exam)


How does general education assessment benefit the university

How does General Education Assessment Benefit the University?

  • Enhanced public relations and visibility

    • Can communicate information to parents, possible applicants, stakeholders and say “this is what a students will know and be able to do as a result of completing the Gen Ed courses at JMU”

  • Enhanced grant funding opportunities

  • Marketable assessment tools can be a source of income


Jmu s marketable assessment tools

JMU’s Marketable Assessment Tools

  • Computer-Based Multiple-Choice Tests

    • ILT: Information Literacy Test (60 items)

    • QR: Quantitative Reasoning Test (45 items)

    • SR: Scientific Reasoning Test (80 items)

  • Marketable Tests Being Developed

    • Oral Communication

    • Critical Thinking


What are other institutions doing

What are Other Institutions Doing?

  • Approaches to Gen Ed assessment (Palomba & Banta, 1999) :

    • Individual Course-Based Approach

    • Multi-Course Approach

    • Noncourse-Based Approach


Individual course based approach

Individual Course-Based Approach

  • Instructor submits a document showing how goals are being addressed in their particular course

    • May include syllabus, quizzes, tests, assignments, activities, examples of students’ work

    • May be judged by other faculty for the extent to which course is covering goals


Individual course based approach1

Individual Course-Based Approach

  • College of William and Mary is taking this approach

  • Also using faculty and student surveys

  • Different Gen Ed “clusters” assessed in different years

  • Office of Assessment oversees process

http://www.wm.edu/wmoa/index.htm


Individual course based approach2

Individual Course-Based Approach

  • Advantages

    • Maximizes faculty involvement

    • Assessment results may help identify changes that need to be made in a particular course

    • May ensure that goals are being addressed in the classroom

  • Disadvantages

    • Can be labor intensive for faculty

    • Doesn’t necessarily involve faculty collaboration

    • Faculty may only choose information that makes them “look good”

    • Not necessarily measuring whether or not student has mastered the goal – just measuring whether or not student was exposed to material that would allow them to master the goal

Some information from Palomba & Banta, 1999


Multi course approach

Multi-Course Approach

  • Faculty from different disciplines join together to decide upon assessment for their “cluster”

  • Winthrop University takes this approach

    • Collect assessment data via course-embedded assessments: common essays, tests, assignments across courses that are graded according to a common rubric

    • Also administers the National Survey of Student Engagement (NSSE) and the College BASE to freshmen and seniors (repeated measures)

    • Office of Assessment oversees process

Some information from Palomba & Banta, 1999

http://www.winthrop.edu/assessment/gened.htm


Multi course approach1

Multi-Course Approach

  • Advantages

    • Encourages faculty collaboration

    • Focuses on student experience in a “cluster”, not in a particular course

    • Shows evidence of student learning

  • Disadvantages

    • When course-embedded, dictates what assignments/tests, etc. must be administered in a particular course, which may be a turn-off to some instructors


Noncourse based approach

Noncourse-Based Approach

  • Campus-wide assessment activities focused on individual student or groups of students, not a particular course

  • Assessments administered outside classroom to all students or a sample or students

  • Assessment tools may be locally-developed (like JMU) or standardized; may include surveys


Noncourse based approach1

Noncourse-Based Approach

  • Advantages

    • Assessment removed from classroom, instructors are given more freedom

    • Faculty collaboration encouraged

    • Focuses on students’ experience in a “cluster”

  • Disadvantages

    • Logistics of collecting data

    • Student motivation to take assessment seriously

    • Mechanisms need to be in place:1) to ensure that students are exposed to materials within courses that will enable them to meet objectives; 2) to match instrument to objectives


Assessment information on internet

Assessment Information on Internet

  • James Madison University

    • http://www.jmu.edu/assessment/

  • University of North Texas

    • http://www.unt.edu/vpaa/Assessment/Assessment_Home.html

  • American Association of Higher Education

    • http://www.aahe.org/

  • National Center for Postsecondary Improvement

    • http://www.stanford.edu/group/ncpi/index.html

  • NC State: Internet Resources for Higher Education Outcomes Assessment

    • http://www2.acs.ncsu.edu/UPA/assmt/resource.htm


  • Login