1 / 39

Program Level Assessment

Program Level Assessment. Workshop for College of the Redwoods Fred Trapp August 18, 2008. Assessment of Program Effectiveness. What the program will do or achieve Curriculum Retention Graduation Placement Satisfaction (graduate and employer). Assessment of Student Learning Outcomes.

dalia
Download Presentation

Program Level Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Program Level Assessment Workshop for College of the Redwoods Fred Trapp August 18, 2008

  2. Assessment of Program Effectiveness • What the program will do or achieve • Curriculum • Retention • Graduation • Placement • Satisfaction (graduate and employer)

  3. Assessment of Student Learning Outcomes • What students will do or achieve • Direct methods of assessment

  4. Direct Measures of Learning • Capstone experience • Standardized tests • Performance on national licensure certification or professional exams • Locally developed tests • Essay questions blind scored by faculty • Juried review of projects • Externally reviewed exhibitions performances • Evaluation of internships based upon program learning outcomes

  5. Indirect Measures of Learning • Alumni, employer, and student surveys (including satisfaction surveys) • Exit interviews of graduates and focus groups graduate follow up studies • Retention and transfer studies • Length of time to degree • Graduation and transfer rates • Job placement rates

  6. Steps of Implementation • Step One: Developing Assessment Plans • Step Two: Developing Implementation Strategies • Step Three: Collecting Data, Analyzing & Reflecting on its meaning • Step Four: Implementing Change as the result of assessment

  7. Assessment Language • Program Outcomes/ Topic Areas • Student Learning Outcomes • Means of Assessment/ Evidence • Criteria for Assessment • Standards of Success/ Quality of Performance

  8. Program Outcome Topic & Student Learning Outcomes Communication (program outcome topic) Write Relate Speak Listen Participate Student learning outcomes

  9. Means of Assessment/Evidence- Circumstances (learning events) • Assignments (in class and out of class) • Feedback on practice • Self evaluation • Peer evaluation • Role Play • Pre Tests • Simulation

  10. Means of Assessment/Evidence- Deliverable or Performance Goal Outcome Outcome Outcome Outcome Outcome Learning events Object (deliverable or performance)

  11. Means of Assessment/Evidence- Deliverable or Performance Communication Write Relate Speak Listen Participate Speech Essay Interview

  12. Criteria for Assessment Communication Write Relate Speak Listen Participate criteria Verbal delivery Speech Nonverbal delivery Organization Content

  13. Standards of Success/ Quality of Performance Communication Write Relate Speak Listen Participate Quality of Performance Verbal delivery Speech Nonverbal delivery organization content

  14. Student Learning Outcomes • Students should be able to <<action verb>> <<something>> • Learner Centered • Specific • Action oriented • Cognitively appropriate for the program level

  15. COMPREHENSION EVALUATION APPLICATION ANALYSIS SYNTHESIS KNOWLEDGE Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe ProducePropose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write

  16. COMPREHENSION EVALUATION APPLICATION ANALYSIS SYNTHESIS KNOWLEDGE Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe ProducePropose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Course outcomes

  17. COMPREHENSION EVALUATION APPLICATION ANALYSIS SYNTHESIS KNOWLEDGE Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe ProducePropose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Program outcomes

  18. Possible Outcomes? • Students will appreciate…. • Students will be exposed to…. • Students will demonstrate knowledge of the history, theories and applications of… • Students will attend…. • Faculty will provide students with opportunities to….

  19. Possible Outcomes • Students will appreciate… • Students will be exposed to…. • Students will demonstrate knowledge of the history, theories and applications of… • Students will attend…. • Faculty will provide students with opportunities to….

  20. Writing Program Level Student Learning Outcomes • Student Learning Outcomes • Students will recognize, analyze, and interpret human experience in terms of personal, intellectual, and social contexts. • Students should be able to critically comprehend, interpret, and evaluate written, visual, and aural material. Compound outcome statements will be problematic to assess.

  21. Program Level Student Learning Outcomes • Students will be able to: • Design a solution to a technical problem • Interpret performance data • Demonstrate the functions of key operating systems • Articulate the ethical standards of the profession • Produce diagnostic quality radiographic images • Evaluate the advantages of alterative solutions

  22. Course LevelStudent Learning Outcomes • Students will be able to: • Students will have a basic knowledge of human anatomy • Students will be able to define 200 medical terms • Students should know the history of the United States • Students will be able to translate dollars into pounds and Euros

  23. Single course Course concepts Fewer confounds from outside the course detractors Faculty (indiv. & small groups) “Micrometer” precision Less fear generated in faculty Faculty professional obligation, less overhead cost No external certifications Multiple courses Program performance competencies (post grad) More confounds from outside the course detractors Faculty (small & large groups) “Two sticks & a chain” precision Potential fear generated in faculty Faculty professional obligation, but some overhead cost External certifications possible Course vs. Program Assessment

  24. Identify Criteria for Assessing Program Level Outcomes • Must be shared across faculty • A way to further define the student learning outcome • Provides a common language for describing student learning • Must be outcome specific • Number of criteria will vary by outcome

  25. Means of Assessment/Evidence- Deliverable or Performance + Criteria Communication Write Relate Speak Listen Participate Eulogy Sales presentation delivery content organization Oral Report

  26. The Reality of Assessing Program Level Student Learning Outcomes Course Course Course Course Course Speaking eye contact gestures volume sources transitions style rate poise examples verbal variety appearance evidence conclusion organization attention getter

  27. The Reality of Assessing Program Level Student Learning Outcomes Can our students deliver an effective public speech? eye contact gestures volume sources transitions style rate poise examples verbal variety appearance evidence conclusion organization attention getter

  28. Program Learning Outcome Example #1 Gather factual information and apply it to a given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected BETTER: Students will be able to apply factual information to a problem Criteria: Relevance Clarity Comprehensiveness Aware of Bias

  29. Program Learning Outcome Example #2 Imagine and seek out a variety of possible goals, assumptions, interpretations, or perspectives which can give alternative meanings or solutions to given situations or problems BETTER: Students will be able to provide alternative solutions to situations or problems Criteria: Variety of assumptions, perspectives, interpretations Analysis of comparative advantage

  30. Program Learning Outcome Example #3 Formulate and test hypotheses by performing laboratory, simulation, or field experiments in at least two of the natural science disciplines (one of these experimental components should develop, in greater depth, students’ laboratory experience in the collection of data, its statistical and graphical analysis, and an appreciation of its sources of error and uncertainty) BETTER: Students will be able to test hypotheses. Criteria: Data collection Statistical Analysis Graphical Analysis Identification of sources of error

  31. Standards for Success/Quality of Performance • Rubric • Scale or description for assessing each of the criteria • Two to Five-point scales for each criteria • Anchored with descriptions and supported by examples • Portion of the Whole Group Which is Successful

  32. Standards for Success/Quality of Performance Rubric OUTCOME: Performance Anchors: Does not meet Expectations Meets Expectations Exceeds Expectations Criteria Descriptions of Performance

  33. Standards for Success/Quality of Performance Rubric OUTCOME: Performance Anchors: Not Yet Almost YES!!!! Components

  34. Standards for Success/Quality of Performance Rubric OUTCOME: Performance Anchors: Basic Essential Exemplary Components

  35. Student Learning Outcome Criteria Test Questions 1 1 2 3 4 2 5 6 7 8 9 3 10 11 12 4 13 14 15 16 17 18

  36. Student Learning Outcome Criteria Test Questions % correct 1 1 2 3 4 78% 2 5 6 7 8 9 56% 3 10 11 12 82% 4 42% 13 14 15 16 17 18

  37. Student learning outcomes Course 1 Course 2 Course 3 Course 4 Course 5 1 X X X 2 X X X 3 X X X X 4 X X 5 X X X X X 6 X X X 7 X Identify Assessment Points in the Program

  38. Phase 1 Student learning outcomes Course 1 Course 2 Course 3 Course 4 Course 5 1 X X X 2 X X X 3 X X X X 4 X X 5 X X X X X 6 X X X 7 X

  39. Communicate Effectively Demonstrate Oral Communication Skills Program-wide Assessment Report Does not meet Meets Exceeds Verbal Delivery 20% 65% 15% 57% 30% Nonverbal Delivery 13% 58% Organization 24% 18% Is there a problem with performance in any of the criteria?

More Related