Creating Effective Outlines and Assessments

1. Creating Effective Outlines and Assessments Dr. Gosha Zywno, Faculty Associate, LTO 3M National Teaching Fellow New Faculty Orientation August 21, 2012

2. Outlines at Ryerson must contain (see Policy 145 for more detail): • Course name, course number, semester and year • Prerequisites and exclusions • Professors name, contact information, office hours • Method of posting grades • Use of email instruction for faculty communication • Course calendar description • Course objectives and focus • Topics and schedule of topics • Type of teaching methods • Outside class activities, if any • Requirements for submission of work • Plagiarism detection tools used (Academic Integrity Office has a proper statement)- deadlines for choosing not to be included in use of these tools • Field placement and technology requirements • Policy on use of electronic equipment in class • Policy on missed work

3. Good course outlines consider instructional design • All components of the course should consider the learning objectives • This includes the course content, assessments, activities, feedback to students • Outlines in different fields are slightly different (you will have a chance to view some from different fields)

4. Good course outlines consider instructional design The goals/objectives of the course are explicitly stated Example of learning objectives: • To become familiar with the basic terms, concepts, principles, and techniques of research methods and statistics • To appreciate the relations between the hypotheses being tested (question asked) and the most appropriate statistical test to use • To become proficient in using SPSS software to conduct various forms of basic data analysis • To feel equipped by term’s end to apply the knowledge and skills gained to better understanding the statistical methods and results in published literature and to conduct one’s own analyses • To understand how to interpret the results from these analyses

5. Alignment What are the desired learning outcomes? “Writing Learning Outcomes,” British Columbia Institute of Technology Learning Resources Unit, https://helpdesk.bcit.ca/fsr/teach/courseprep/htoutcomes.pdf

6. Bloom’s Taxonomy • Benjamin Bloom (1956), identified three domains of educational activities: • Cognitive: mental skills (Knowledge) • Affective: growth in feelings or emotional areas (Attitude) • Psychomotor: manual or physical skills (Skills)

7. Think-Pair-Share Activity • Reflect individually on the topic (THINK) • Turn to your neighbour (PAIR) • Logistics: introduce yourself, decide Recorder-Reporter roles • Recorder – your birth date is closest to today: take notes of your discussion • Reporter – you may be called on to report to the large forum • Take 5 minutes to discuss/record the topic • Report back to the larger group (SHARE) • Some pairs may be called on to report results Total time: 15 minutes

8. Think-Pair-Share Activity • Logistics: • Think (individual) – 90 sec • Pair (discuss) – 5 minutes • Share (large group) – 5 minutes • Topic: • In a course you will be teaching, identify one learning objective in each of the three domains (cognitive, affective, psychomotor) Total time: 15 minutes

9. Cognitive Outcomes Old Version New Version http://www.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm http://www.officeport.com/edu/blooms.htm Decide at what cognitive level you want to accomplish your goals: Different levels of cognitive learning Bloom’s Taxonomy

10. Bloom’s Taxonomy Example: ??? To measure knowledge(common terms, routine info, definitions, descriptions, facts, principles, procedures), ask these kinds of questions: Define, Describe, Identify, Label, List, Match, Name, Outline, Reproduce, Select, State. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

11. Bloom’s Taxonomy List steps involved in titration To measure knowledge(common terms, routine info, definitions, descriptions, facts, principles, procedures), ask these kinds of questions: Define, Describe, Identify, Label, List, Match, Name, Outline, Reproduce, Select, State. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

12. Bloom’s Taxonomy Example: ??? To measure comprehension(understanding of facts and principles, interpretation of material), ask these kinds of questions: Convert, Defend, Distinguish, Estimate, Explain, Extend, Generalize, Give examples, Infer, Predict, Summarize. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

13. Bloom’s Taxonomy Summarize basic tenets of deconstructionism To measure comprehension(understanding of facts and principles, interpretation of material), ask these kinds of questions: Convert, Defend, Distinguish, Estimate, Explain, Extend, Generalize, Give examples, Infer, Predict, Summarize. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

14. Bloom’s Taxonomy Example: ??? To measure application(solving problems, applying concepts and principles to new situations, using rules to perform calculations), ask these kinds of questions: Demonstrate, Modify, Operate, Prepare, Produce, Relate, Show, Solve, Use. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

15. Bloom’s Taxonomy Calculate the deflection of a beam under uniform loading To measure application(solving problems, applying concepts and principles to new situations, using rules to perform calculations), ask these kinds of questions: Demonstrate, Modify, Operate, Prepare, Produce, Relate, Show, Solve, Use. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

16. Bloom’s Taxonomy Example: ??? To measure analysis(recognition of unstated assumptions or logical fallacies, ability to distinguish between facts and inferences, breakdown of problems to see structure, determine properties, see connections), ask these kinds of questions: Diagram, Differentiate, Distinguish, Illustrate, Infer, Point out, Relate, Select, Separate, Subdivide. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

17. Bloom’s Taxonomy In the speech discussed, which statements are based on facts and which are based on assumptions? To measure analysis(recognition of unstated assumptions or logical fallacies, ability to distinguish between facts and inferences, breakdown of problems to see structure, determine properties, see connections), ask these kinds of questions: Diagram, Differentiate, Distinguish, Illustrate, Infer, Point out, Relate, Select, Separate, Subdivide. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

18. Bloom’s Taxonomy Example: ??? To measure synthesis(integrate learning from different areas or solve problems by creative thinking, design, form a new system or solution), ask these kinds of questions: Categorize, Combine, Compile, Devise, Design, Explain, Generate, Organize, Plan, Rearrange, Reconstruct, Revise, Tell. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

19. Bloom’s Taxonomy How would you restructure the school day to reflect children's developmental needs? To measure synthesis(integrate learning from different areas or solve problems by creative thinking, design, form a new system or solution), ask these kinds of questions: Categorize, Combine, Compile, Devise, Design, Explain, Generate, Organize, Plan, Rearrange, Reconstruct, Revise, Tell. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

20. Bloom’s Taxonomy Example: ??? To measure evaluation (judging and assessing), ask these kinds of questions: Appraise, Compare, Conclude, Contrast, Criticize, Describe, Discriminate, Explain, Justify, Interpret, Support. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

21. Bloom’s Taxonomy Why is Bach's Mass in B Minor acknowledged as a classic? To measure evaluation (judging and assessing), ask these kinds of questions: Appraise, Compare, Conclude, Contrast, Criticize, Describe, Discriminate, Explain, Justify, Interpret, Support. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

22. Bloom’s Taxonomy Kelley, K. & McDonald, R. Information Literacy and Writing Assessment Project: Tutorial for Developing and Evaluating Assignments. University of Maryland University College. http://www.umuc.edu/library/libhow/informationliteracy_tutorial.cfm

23. http://www.fgcu.edu/onlinedesign/designDevd.html

24. Alignment Design assignments (and activities) to align with your desired course outcomes; some questions to ask: • What are the outcomes to be assessed? • What are the capabilities/skills (implicit or explicit) in the outcomes? • Does the specific assessment task match the outcomes and skills? • Is the method efficient in terms of student time and staff time? • What alternatives are there? What are their advantages and disadvantages? • Are the marking schemes or criteria appropriate? Brown, G. (2001). Assessment: A Guide for Lecturers. The LTSN Generic Centre Assessment Series. http://www.heacademy.ac.uk/resources/detail/assessment/assessment_series

25. Effective Assessments - Summary • Align with Learning Objectives & Activities • Match appropriate level of Cognitive Learning • Have clear expectations • Have reasonable workloads • Engage students with material, are meaningful • Have fair and consistent grading - use rubrics

26. Effective Assessments - Summary • Align with Learning Objectives & Activities • Match appropriate level of Cognitive Learning • Have clear expectations • Have reasonable workloads • Engage students with material, are meaningful • Have fair and consistent grading - use rubrics • Encourage Academic Integrity • Support Information Literacy • Support EAL Students • Are Accessible (Universal Instructional Design)

27. Information Literacy An information literate student is able to: • Determine the extent of information needed • Access the information effectively and efficiently • Evaluate information and its sources critically • Use information effectively to accomplish a specific purpose • Understand the economic, legal and social issues surrounding the use of information, and access and use information ethically and legally

28. Information Literacy Information literacy-based assessments: • Encourage self-directed learning and critical thinking • Favor analysis and interpretation over rote-learning • Require research and evaluation on a variety of resources • Provide concrete feedback on progress

29. Effective Assessments Information Literacy Example • Follow a research trend. • Select a present day issue or concern and then search the past literature at five or ten year intervals. • Discuss how theories or attitudes have endured or changed. Touro College Libraries. http://www.tourolib.org/services/information-literacy/assignments

30. Avoid cheating and plagiarism: • Make assignments unique and personal • Vary assignments and rotate tests • Request rough drafts and description of research methods, or assign work in steps • Be specific about grading criteria, consistent in grading For more information, visit: http://www.ryerson.ca/academicintegrity/

31. Make assignments as specific as possible: • Use more obscure material • Pose a more focused question • Require application rather than explanation • Consider a tight comparison • Make assignments unique and personal, change often Discouraging Plagiarism. Center for Innovative Teaching and Learning, Indiana University Bloomington. http://www.iub.edu/~citl/services/writing/plagiarism.php

32. Effective Assessments Academic Integrity Example • Rather than: Explain the basic functions of the vascular, skeletal, muscular and nervous systems • Try: A cat jumps off the end of a table onto the floor. Describe how its vascular, muscular, skeletal and nervous systems contribute to this action. Discouraging Plagiarism. Center for Innovative Teaching and Learning, Indiana University Bloomington. http://www.iub.edu/~citl/services/writing/plagiarism.php

33. Break down assignments • Gives students time to ask questions • Encourage a simple, direct style • Emphasize ‘process’ rather than ‘product’ • Incorporate low-stakes writing assignments • Provide models • Use explicit rubrics http://www.ryerson.ca/studentservices/els/

34. Effective Assessments EAL Learners Example • Use familiar, high frequency words, rather than low frequency, academic or idiomatic language • Rather than:“Mitotic cell division is initiated in the ______.” • Try:“Mitotic cell division begins in the ______.” Teemant, A. (2010). ESL Student Perspectives on University Classroom Testing Practices. Journal of the Scholarship of Teaching and Learning, 10(3), 89-105.

35. Accessibility Universal Instructional Design creates barrier free assignments that benefit all students • #1: Design instructions with the user in mind • Present information in small chunks separated by headers • Put key tasks and dates in bold • Use a large, clear font   • #2: Reduce barriers to accessing the assignment • Provide access to instructions and supplemental material in multiple formats Four Ways to Incorporate Universal Design for Learning into Your Assignment Instructions http://gwired.gwu.edu/dss/Newsletters/Fall05UDL/

36. Accessibility Universal Instructional Design creates barrier free assignments that benefit all students • # 3 List assignment goals and evaluation criteria from the start • With focus on objectives, it is easier to be flexible without losing academic rigour • # 4 Provide Examples • Show different approaches to the assignment • Get permission to share examples of previous students’ work • Post samples in an accessible electronic format on Blackboard or class website Four Ways to Incorporate Universal Design for Learning into Your Assignment Instructions http://gwired.gwu.edu/dss/Newsletters/Fall05UDL/