Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Ass...
Download
1 / 44

Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Assignments - PowerPoint PPT Presentation


  • 311 Views
  • Uploaded on

Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Assignments. Dr. Blase Scarnati Chair, Liberal Studies Committee Associate Professor, Musicology, School of Music

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Assignments' - RexAlvis


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Assignments

Dr. Blase ScarnatiChair, Liberal Studies CommitteeAssociate Professor, Musicology, School of Music

Dr. Cynthia ConnAssociate Director, Office of Academic AssessmentClinical Instructor, Educational Technology Program

Dr. Bruce FoxChair, Liberal Studies Committee Assessment SubcommitteeDirector, Honors Program


Agenda l.jpg
Agenda for Purposes of Identifying Embedded Assignments

  • Overview

  • Definition of Skills

  • Pilot Study

  • Content Analysis

  • Group Discussion: Ideas, Projects, and Challenges


Northern arizona university l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Northern Arizona University

  • Flagstaff, Arizona

  • State supported

  • Doctoral intensive

  • Significant distance learning


Northern arizona university4 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Northern Arizona University

  • 19,000 total students

  • Approximately 6,000 graduate students (3,800 FTE)

  • Approximately 5,500 students statewide


Northern arizona university5 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Northern Arizona University

  • Approximately 11,400 undergraduates on the Flagstaff campus

  • Students come to us in a variety of ways


Liberal studies program l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Liberal Studies Program

  • Approved Spring 1998

  • Fully implemented Fall 1999


Citizenship l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Citizenship

  • Coherent program

  • Essential skills necessary for citizenship

  • Key themes


Liberal studies program requirements l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Liberal Studies Program Requirements

  • 35 credit total requirement

  • Foundation requirements

  • Distribution courses

  • Courses embedded within the academic major


Foundation requirements l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Foundation Requirements

  • English composition course (4 credits)

  • Mathematics foundation course (3 credits)


Distribution requirements 25 credits l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Distribution Requirements (25 Credits)

  • Aesthetic and Humanistic Inquiry (3-6 credits)

  • Cultural Understanding (3-6 credits)

  • Laboratory Science (4 credits)

  • Science and Applied Science (3-6 credits)

  • Social and Political Worlds (3-6 credits)


Thematic foci l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Thematic Foci

  • Environmental Consciousness

  • Technology and Its Impact

  • Valuing the Diversity of Human Experience


Essential skills l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Essential Skills

  • Effective oral communication

  • Critical reading

  • Effective written communication

  • Quantitative analysis and/or spatial reasoning

  • Creative thinking

  • Critical thinking

  • Scientific inquiry

  • Ethical reasoning

  • Effective use of technology


University requirements l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

University Requirements

  • Courses embedded within the academic major

  • Diversity requirement

  • Required freshman colloquium


In the academic major l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

In the Academic Major

  • Junior level writing course (3-4 credits)

  • Senior capstone experience (3 credits minimum)


Diversity requirement l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Diversity Requirement

  • U.S. Ethnic Diversity (3 credits)

  • Global Awareness (3 credits)


Required freshman colloquium 3 credits l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Required Freshman Colloquium (3 Credits)

  • Discontinued effective Fall 2004

  • Redesigned


Liberal studies committee lsc l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Liberal Studies Committee (LSC)

  • Faculty body

  • Six colleges

  • Student representative

  • Faculty Senate representative

  • Administrative and programmatic representatives


Assessment l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Assessment

  • Liberal Studies Committee Assessment Subcommittee

  • Liberal Studies Program Assessment Plan (Fall 2004)


Assessment plan goals 1 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Assessment Plan Goals (1)

  • Assess the achievement of NAU students

  • Effective writing

  • Effective oral communication

  • Critical reading


Assessment plan goals 2 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Assessment Plan Goals (2)

  • Create a more cohesive curriculum of study

  • Analytical procedures

  • Inform adjustments to the curriculum


Research question 1 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Research Question 1

  • To what extent are courses and embedded assignments that state they teach effective writing, critical reading, and/or effective oral communication aligned with the outcomes defined during the 2003-2004 academic year by the Liberal Studies Committee?


Research question 2 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Research Question 2

  • How should issues of non-alignment between courses and defined outcomes be addressed?


Research question 3 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Research Question 3

  • What expectations should be established for course content, assessment, and the broader curriculum?


Research question 324 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Research Question 3

  • Specifically, to enhance the cohesiveness of the curriculum, what changes should be made to the existing expectations (for faculty, departments, etc.) for submission and acceptance of courses into the Liberal Studies Program?


Assessment audit l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Assessment Audit

  • Courses and embedded assignments

  • Aligned with the outcomes

  • Pilot Project


Defining skills l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Defining Skills

  • Working definitions created by LSC subcommittee

  • Working definitions tentatively approved by entire LSC

  • Definitions “vetted” through consultation with internal constituents, primarily departments

  • Consensus definitions approved by LSC


Defining skills27 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Defining Skills

Agenda

Overview

Skills

Pilot Study

Content

Analysis

Group

Discussion

  • Advantages:

    • Create buy in from other campus units

    • Liberal Studies process seen as open and participative

    • Better definitions!

  • Key strategy: visit folks personally. Do NOT rely solely on electronic communication


Effective writing l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Effective Writing

  • Students will be able to:

    • Demonstrate steps in the writing process

    • Tailor writing to a specific audience

    • Focus writing on a specific purpose

    • Provide logically coherent pieces of written work

    • Apply general writing standards


Critical reading l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Critical Reading

  • Students will be able to:

    • Summarize accurately and comprehensively

    • Recognize the most significant textual and visual elements

    • Incorporate/use specific examples and/or quotations

    • Recognize patterns and valid comparatives

    • Demonstrate an understanding/comprehension of the material

    • Demonstrate evidence of an ability to analyze and evaluate texts

    • Demonstrate the ability to explicitly identify criteria and use these criteria accurately to evaluate a piece of written work


Oral communication l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Oral Communication

  • Students will be able to:

    • Organize and deliver content based on audience, location, allotted time, and purpose

    • Speak with appropriate oral communications techniques

    • Listen and respond thoroughly and thoughtfully to questions

    • Create appropriate professional supplemental materials that reinforce the presentation


Quantitative spatial analysis l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Quantitative & Spatial Analysis

Agenda

Overview

Skills

Pilot Study

Content

Analysis

Group

Discussion

(aka “Methods of Formal Analysis”)

(aka “Quantitative Literacy”)

Suggested titles: More inclusive?


Methods of formal analysis l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Methods of Formal Analysis

Agenda

Overview

Skills

Pilot Study

Content

Analysis

Group

Discussion

  • Application of numerical, visual or symbolic perspectives for the purpose of drawing inferences, understand phenomena or making predictions in situations that involve natural phenomena or real-world decision making


Pilot project l.jpg
Pilot Project for Purposes of Identifying Embedded Assignments

Agenda

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

  • Testing of Protocols

  • ITERATIVE

  • Foundation Principles

    • Based on embedded assignments

    • Actively involves faculty

    • Supplements direct evidence with indirect date sources


Pilot project34 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Pilot Project

  • Process

    • Identify faculty teaching courses that include desired skill

    • Recruit these faculty as sources of embedded assignments

    • Use faculty consultants to develop and evaluate rubrics

    • Use rubrics to assess level of skill mastery


Pilot project35 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Pilot Project

  • Process

    • Collect assessment data for a stratified sample of courses

    • Compile and analyze data

    • Triangulate using indirect data sources

    • Evaluate protocols


Pilot project36 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Group Discussion

Pilot Project

  • Informing and Involving the Wider Campus Community

    • Communications Plan Strategy to inform entire campus community of assessment process in a deliberate and on-going manner

    • Create a Steering Committee of on and off campus constituencies


Content analysis l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Small Discussion

Full Group Discussion

Agenda

Overview

Skills

Pilot Study

Content

Analysis

Group

Discussion

Content Analysis

  • Purpose

    • Identify the alignment between Liberal Studies skills as they are now defined and syllabi of current Liberal Studies courses.

    • Identify embedded assignments that could be used for assessment purposes


Content analysis38 l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content

Analysis

Group

Discussion

Content Analysis

  • How will findings be used?

    • Identify potential faculty for participation as consultants for Pilot Study

    • Target courses with strongest alignment for first phase of data collection

    • Recommend policy changes regarding content of syllabi and expectations related to assessment


Group discussion l.jpg
Group Discussion for Purposes of Identifying Embedded Assignments

Agenda

Overview

Skills

Pilot Study

Content

Analysis

Group

Discussion

  • Ideas: What works well at your institution with respect to assessing General Education student learning outcomes?

  • Challenges: What challenges concerning General Education and Assessment do you face at your institution?


References l.jpg

Agenda for Purposes of Identifying Embedded Assignments

Overview

Skills

Pilot Study

Content Analysis

Small Discussion

Full Group Discussion

References

  • Walvoord, Barbara E. (2004). Assessment Clear and Simple : A Practical Guide for Institutions, Departments, and General Education. San Francisco: Jossey-Bass.

  • Neuendorf, Kimberly A. (2002). The Content Analysis Guidebook. Thousand Oaks: Sage Publications.


Contact information l.jpg
Contact Information for Purposes of Identifying Embedded Assignments

Blase Scarnati, Ph.D.

Chair, NAU Liberal Studies CommitteeAssociate Professor, School of MusicP.O. Box 6040Flagstaff, Arizona 86011

[email protected]

(928) 523-6537

Cynthia Conn, Ph.D.

Associate Director, Office of Academic AssessmentClinical Instructor, Educational Technology ProgramP.O. Box 4091Flagstaff, Arizona 86011

[email protected]

(928) 523-7624

Bruce Fox, Ph.D.

Chair, Liberal Studies Committee Assessment Subcommittee

Director, Honors ProgramP.O. Box 5689Flagstaff, Arizona 86011

[email protected]

(928) 523-6636


ad