Assessing Graduate Program Curriculum Renewal in

1. COLLEGE of FOOD, AGRICULTURAL, and ENVIRONMENTAL SCIENCES Assessing Graduate Program Curriculum Renewal in Horticulture and Crop Science & Engineering Education FRIDAY, February 1, 2019 Michelle L. Jones, Ph.D. Ann D. Christy, PhD, PE

2. COLLEGE of FOOD, AGRICULTURAL, and ENVIRONMENTAL SCIENCES • Case study: New program (EED) • How to determine if the time is right to undertake a holistic revision (group discussion) • Case study: Program renewal (HCS) • Lessons Learned • Q & A Overview / Agenda

3. Case Study: New Program • Engineering Education Ph.D. program • Overall vision: interdisciplinary, research focused, doctoral program that combines disciplines of engineeringand education leading to multiplealumni career paths: • Academia • Industry (Chief Learning Officers, eLearning specialists) • Not-for-profit / informal technical education • K-12 and community college collaboration • Government / policy

4. Engineering Education PhD proposal developed over a 9 month-long process by our grad studies committee and guided by Dr. Alan Kalish and Dr. Teresa Johnson of UITL / UCAT (June 2016 start)

5. Process that engaged stakeholders • Curriculum design starting with learning goals and objectives • Open, collaborative communication and feedback strategy (for establishing both the program and the department) • Wiki site: open to all OSU Carmen Wiki users who could visit the site, read the latest revision, and add comments:https://carmenwiki.osu.edu/display/10700/EEIC+Department+proposal+Home • In-person presentations at college faculty meetings and beyond

6. Five program goals, • developed by a team of faculty (graduate studies committee plus), and each goal has 4 to 10 associated outcomes. • The successful engineering education doctorate graduate will be able to…

7. 1. Identify, discuss, and address critical issues facing engineering education in alignment with stakeholder needs

8. 2. Design, conduct, and critique research in engineering education

9. 3. Demonstrate, value, and apply engineering expertise

10. 4. Create, teach, and assess courses and curricula

11. 5. Identify, demonstrate, and value appropriate personal and professional skills, mindsets, and traits

12. Each outcome has 3 levels of proficiency: • Basic • Intermediate • Advanced • …and these 3 levels of proficiency are mapped across the curriculum for each course and co-curricular milestone

13. Curriculum map excerpt: specialization

14. Brainstorming assignments to assess proficiency in learning outcomes, as a basis for building courses…

15. Course-based curriculum • Required Courses in Engineering Education (26 cr. hrs.) • 4 core courses (Foundations, Learning theory / pedagogy / assessment, Research design, Professional development) • 3 research methods courses (quantitative & qualitative) • 2 teaching practicum courses (I & II) • 2 seminars • Additional Required Courses • Specialization (12 cr. hrs.) • Disciplinary engineering at graduate level (12 cr. hrs.) • Dissertation Research (30 cr. hrs.)

16. ENGR EDU 6200: Learning Theory, Pedagogy, and Assessment An example of our ENGREDU course design: Always start with learning outcomes!

17. ENGR EDU 7900: Professional Development in Engineering Education

18. Co-Curricular elements • Admission requirements • Essay prompts and prior experience • Engineering portfolio and/or M.S. engineering degree • Graduate program milestones • Qualifying exam • Proposal defense and Candidacy exam • Annual review – individual learning experiences • Reflection and feedback on teaching opportunities & research group interactions, dissertation progress, specialization activities • Dissertation document and oral defense

19. Qualifier Exam • Designed the qualifier to test each of the outcomes assigned to it in the curricular map • Written portion essay prompts (often verbatim proficiencies) • Oral portion team assignment allowed faculty to observe students interact (scoring their teamwork and communication) • Exam Structure • Foundations: Review NSF program solicitation • Research design: Review journal article • Pedagogy: Review undergraduate self study report, course syllabus, assignments, and lesson plans • Team oral panel: Review / ranking of NSF proposals

20. This was an example of building a new graduate program, beginning with initial UITL/UCAT sessions in June 2016 to final ODHE approval in July 2018. More often, faculty are faced with revising an existing program… • How do you know the time is right to undertake a holistic revision? • Where are you in this process?

21. PhD Curriculum Review- Department of Horticulture and Crop Science Michelle L. Jones, February 1, 2019

22. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES HCS PhD curriculum review and revision – in progress • Process is being driven by an Ad hoc committee led by Dr. Michelle Jones, formerly Chair of the Graduate Studies Committee. • Guided by Dr. Teresa Johnson, University Institute for Teaching and Learning • Started in Summer 2017 (initial meeting with faculty) • Curriculum review committee regular meetings with Teresa Jan 2018

23. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES Reasons for undertaking this review • Current course requirements were implemented in 2008 (minor changes to PhD requirements in 2017) • We made some changes to the courses quarters- semesters (2012) • We have never looked at the content of the courses as an integrated curriculum • We do not have a process for regular review of the learning objectives within our graduate courses • New faculty who are asked to develop a course in their area of expertise must make sure it aligns with the needs of the students and the current curriculum 1

24. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES Reasons for undertaking this review Feed back from students: antidotal, listening sessions, survey of current students, survey of alumni (5 years)

25. Survey of current graduate students

26. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES Challenges • Horticulture and Crop Science is a diverse department with programs from very applied agronomy and horticulture to very basic programs in molecular biology (genetics, genomics and metabolomics). • How do we provide some consistency within the educational experience of PhD students in the program while allowing for flexibility. • How do we create a balance between breadth in plant science knowledge and depth within a discipline (ex. Genetics). • Are we too diverse to have a common core of courses or do we expect all PhD students graduating from HCS to have a similar foundational knowledge of plant science.

27. How we engaged the faculty Aug. 2017 All faculty discussion -What do we want the program to look like? What should the program be known for? Jan. 2018 Brainstorm program goals – all faculty April 2018 Review of goals and outcomes with faculty Sept. 2018 Review of goals, outcomes & proficiencies with faculty

28. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES Current HCS PhD Program All PhD students in HCS are required to complete a curriculum of courses that encompass crop physiology, ecology, plant breeding & biotechnology, and experimental design. HCS 5621 or 7821 Physiology HCS 7625 or 8825 Plant Breeding & Biotechnology HCS 5602 Ecology HCS 8887 Experimental Design HCS 7001 Professional Development HCS 7890 Colloquium (written research proposal and oral presentation) HCS 8830 Current Topics (x3) HCS 7806 Research Methods (x2)

29. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES Goals – HCS PhD Program Goal A: Analyze, interpret and apply foundational knowledge of plant and related sciences to discovery and problem solving (Foundational Knowledge). Goal B: Use the scientific method to design, conduct, and evaluate research that makes creative, theoretical and practical contributions to plant science (Research). Goal C: Demonstrate professionalism, leadership, creativity and responsible conduct of research (Professional Development). Goal D: Utilize oral, written, and digital forms of communication to clearly convey disciplinary knowledge, its relevant application, and value to peers, stakeholders, students and the public (Communication).

30. Curriculum map – Goals, Outcomes, Proficiencies (Beginner, Intermediate, Advanced)

31. What have we learned from our curriculum map? What proficiencies are not in any required courses? What proficiencies are in many required courses? Are the advanced proficiencies only in the advanced courses? Are students who chose different courses (i.e. HCS 5621 or 7821) still getting all the proficiencies? Are there gaps and overlaps?

32. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES Challenges - course proficiencies Of our core courses not all students take the same course (i.e. Physiology HCS 5621 or HCS 7821) This has resulted in 4 “options” or combination of the classes – all proficiencies are not met depending on the options the student choses. Physiology HCS 5621 and HCS 7821 - is this beginning level and advanced. HCS 8830 Current Topics and 7806 Research Methods – different topics and course offerings so they can not be mapped.

33. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES Co-curricular activities- How do we assess learning that occurs outside the class room? Annual reviews (by advisors and self reviews) Written and oral candidacy exams Dissertation Organizing the Graduate Student Research Symposium (Leadership) Presenting at the Graduate Student Research Symposium Teaching Innovation???

34. What have we learned - Many of the professional development proficiencies are not assessed in HCS classes Opportunities: Expand the learning outcomes for our professional development class (HCS 7001) Determine if some of these can be added to HCS “core”

35. What have we learned - Many of the communication proficiencies are not assessed in HCS classes Opportunities: Determine if some of these can be added to HCS “core” Develop HCS7806 (or 7890) methods courses that focus on grant proposal writing, oral presentations, extension writing etc.

36. COLLEGE OF FOOD, AGRICULTURAL AND ENVIRONMENTAL SCIENCES What do we hope to get out of this process? A PhD curriculum map that can help guide the development of new courses to fill gaps and meet student needs A curriculum that is flexible and can be adapted to meet changing needs of stakeholders A process to align current courses with the proficiencies agreed upon by the faculty. A process to review and update courses An assessment process that determines what students are learning A plan for using these assessments to continually improve the program.

37. COLLEGE of FOOD, AGRICULTURAL, and ENVIRONMENTAL SCIENCES • Lessons Learned • Team project, contrary to academic norm of course ownership • Non-course elements – qualifying/candidacy exam design • Assessment and closing the loop • Annual course round-up and what to do with proficiencies not covered in a class? • Need to have structured onboarding of new instructors • Amount of work to do this right/well - resourcing faculty time • Not the end point -> starting framework to design the classes that make it happen • Write three levels of proficiency for each outcome, and decide the natural path for students to learn an outcome

38. COLLEGE of FOOD, AGRICULTURAL, and ENVIRONMENTAL SCIENCES • Questions? • Comments? • Suggestions? • Please contact: • Michelle L. Jones <jones.1968@osu.edu> Ann D. Christy <christy.14@osu.edu>