1 / 45

Outcomes Assessment Using Capstone Experiences

This article explores the assessment of capstone experiences in engineering programs, focusing on incorporating as many outcomes as possible. It discusses the desired attributes of capstone experiences, the development of rubrics for assessment, simultaneous assessment and grading, and the use of oral presentations for evaluation.

pamula
Download Presentation

Outcomes Assessment Using Capstone Experiences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Outcomes Assessment Using Capstone Experiences Joseph A. Shaeiwitz Chemical Engineering Department West Virginia University P.O. Box 6102 Morgantown, WV 26506-6102 304-293-2111 ext. 2410 jashaeiwitz@mail.wvu.edu http://www.che.cemr.wvu.edu/ugrad/outcomes/

  2. Assessment Survey • Please complete the pre-workshop assessment survey

  3. Outline • Introduction/Motivation • How to include as many outcomes as possible in a capstone experience • Desired attributes of capstone experiences

  4. Outline (cont’d) • Development of rubrics to assess capstone experiences • Simultaneous assessment and grading • Using oral presentations for assessment of capstone experiences • Summary

  5. Outline • Introduction/Motivation • How to include as many outcomes as possible in a capstone experience • Desired attributes of capstone experiences

  6. Motivation • Self-assessment measures such as surveys are insufficient, assessment by the faculty is needed • All engineering programs already have capstone experiences • Students are expected to apply previously learned knowledge in capstone experiences

  7. Feedback Model education process one class one course graduate alumnus one class entering college one course

  8. Outline • Introduction/Motivation • How to include as many outcomes as possible in a capstone experience • Desired attributes of capstone experiences

  9. Apply math, sci, engr Experiments Design Teams Solve engr problems Ethics Communication Broad education Life-long learning Contemp. Issues Modern tools Required Outcomes

  10. Exercise 1 • Suggest a capstone experience that includes as many required outcomes as is possible • Focus on what needs to be added to commonly used experiences to include additional outcomes

  11. Some Ideas • Comprehensive individual/group design project • Includes all technical aspects • Include material not taught in class • Can include safety, life-cycle analysis, pollution prevention, environmental impact statement • Requires use of specialized, sophisticated software

  12. Outline • Introduction/Motivation • How to include as many outcomes as possible in a capstone experience • Desired attributes of capstone experiences

  13. Attributes of Capstone Experiences • What attributes do you expect to see in your capstone experiences? • What skills and/or knowledge do you expect to be demonstrated in capstone experiences?

  14. Exercise 2 • List as many attributes as possible for the capstone experience described in Exercise 1 • Focus on what you would be evaluating when grading the final project report

  15. Some Ideas • Writing – grammar, punctuation, format, good technical explanations, logical development • Oral – use of visual aids, organization, delivery mechanics • Technical – apply math, science, engineering; design of equipment; optimization based on economics

  16. Outline • Development of rubrics to assess capstone experiences • Simultaneous assessment and grading • Using oral presentations for assessment of capstone experiences • Summary

  17. Development of Rubrics • What attributes do you expect to see in your capstone experiences? • What skills and/or knowledge do you expect to be demonstrated in capstone experiences?

  18. Development of Rubrics (cont’d) • How would you describe excellence for each attribute? • What is not acceptable for each attribute? • Fill in intermediate descriptions

  19. Four-Point Scale for Rubrics • 4 – exceeds expectations • 3 – meets expectations • 2 – below expectations • 1 – not acceptable

  20. Use of Rubrics • Oral Reports • Written Reports • Technical Content

  21. Advantages of Method • Can involve many faculty members • Standardization of evaluation • Is done while grading • Can be adapted to guideline for grading

  22. Exercise 3 • For the attributes listed in Exercise 2, develop a rubric for evaluation of these attributes

  23. Outline • Development of rubrics to assess capstone experiences • Simultaneous assessment and grading • Using oral presentations for assessment of capstone experiences • Summary

  24. Primary Trait Analysis • Rubric-like grading guide • Converts “unstated criteria” (e.g., “it feels like a B,” to “highly explicit criteria.” • Converts from norm-referenced to criterion-referenced grading • B. E. Walvoord and V. J. Anderson, Effective Grading. A Tool for Learning and Assessment, Jossey-Bass, 1998.

  25. Our Experiences • Five-point scale is difficult to develop • We use four-point scale shown earlier • Approximate grades • A = several 4, all others 3 • B = all 3 • C = average about 2.5 or so • D = average around 2 • F = average well below 2

  26. Large-Group Design Project • Students work under direction of student chief engineer • Faculty members play roles – one is client, other is student company vice president • Evaluations like in company • Rubric developed for grading

  27. Outline • Development of rubrics to assess capstone experiences • Simultaneous assessment and grading • Using oral presentations for assessment of capstone experiences • Summary

  28. Feedback from Q & A Session • Have presentation followed by question and answer session • Students get immediate feedback • Ask questions and follow-up questions • Look for patterns – common errors or common good ideas

  29. Types of Questions Asked • Can they explain what was done and why? • Quiz students on related or peripheral concepts • Error is observed – convert session into tutorial • How has student applied what was learned?

  30. Information Seeking • Level of understanding and ability to communicate it • Was “right” answer obtained for wrong reason or accidentally? • Is “big picture” understood? • How was solution obtained?

  31. Example – weak student • Student chooses reactor temperature of 325°C • Question: Why was this reactor temperature chosen? • Good: Because it was economic optimum • Bad: I just chose it

  32. Example – better student • Student chooses reactor temperature of 325°C and shows graph illustrating that it is economic optimum • Question: Why is this optimum? Why are economics less favorable above and below this temperature? • Good: Provides explanation • Bad: I just tried a range of temperatures and this is what I found.

  33. Example – very good student • Student chooses reactor temperature of 325°C, shows graph illustrating that it is economic optimum, and explains why • Question: Did you investigate other variables such as pressure, conversion? • Good: Yes, and explains result • Bad: No, I just investigatedtemperature.

  34. Example – excellent student • Student chooses reactor temperature of 325°C, shows graph illustrating that it is economic optimum, explains why, and does same for other decision variables • Question: It becomes more difficult to find good questions

  35. Exercise 4 • Ask me questions about this presentation aimed at revealing my understanding of assessment, ABET EC 2000, rubric development, etc.

  36. Outline • Development of rubrics to assess capstone experiences • Simultaneous assessment and grading • Using oral presentations for assessment of capstone experiences • Summary

  37. Summary • Assessment using capstone experiences can complement surveys – assessment by the faculty • It is possible to include many outcomes in capstone experiences • Identify desired attributes of capstone experiences

  38. Summary (cont’d) • Can develop rubrics to assess capstone experiences • Can use primary trait analysis for simultaneous assessment and grading • Can use oral presentations as part of assessment of capstone experiences

  39. Web Site http://www.che.cemr.wvu.edu/ugrad/outcomes/

  40. Assessment Survey • Please complete the post-workshop assessment survey

More Related