“HELPING STUDENTS LEARN” IS CORE TO NDSCS

1. “HELPING STUDENTS LEARN” IS CORE TO NDSCS Gloria Dohman, Ph.D. Associate Vice President Institutional Effectiveness North Dakota State College of Science

2. Principles of Good Practice for Assessing Student Learning • Begins with educational values • Mission, vision of the school • What do “we” want our students to learn? Why? • What learning activities should “we” observe and measure? How should we observe and measure them?

3. (cont) • Based on clear, explicitly stated purposes • Reflects an understanding of learning as multidimensional, integrated and revealed in performance over time • Equal attention to outcomes and experiences that led to the outcomes • Must be an ongoing, not episodic process

4. (cont) • Fosters wider improvement though involvement of representatives across the educational community • Begins with issues of use and illuminates questions that you really care about • Part of a larger set of conditions that promote change • Through assessment, educators meet responsibilities to students and to the public (adapted from American Association of Higher Education, 1992)

5. NDSCS Mission, Vision, Values • How does NDSCS’s Mission, Vision, and Values affect how we approach student learning? • How is our approach different from other higher education institutions? Or is it? Should it be?

6. “Learning” is both the focus and the goal • What have our students learned? • How well have they learned it? • How successful have we been at what we are trying to accomplish?

7. Questions to ask • What do we most want our students to learn? • Is it the right learning?—Where is our validation? • What and how well are students learning what we intend for them to learn? • What knowledge, skills and abilities should students be able to demonstrate?

8. Assessment Defined • “The process of gathering and discussing information from multiple and diverse sources in order to develop a deep understanding of what students know, understand and can do with their knowledge as a result of their educational experiences; the process culminates when assessment results are use to improve subsequent learning” (Huba and Freed,p. 8).

9. What is Assessment of Student Learning? • A philosophy of education which stresses better understanding of student learning and continuous improvement of instruction • A comprehensive process aimed at understanding and improving student learning • Documentation of proficiencies in skills and competencies according to established student outcomes • An ongoing process to improve curriculum and instruction and promote student learning

10. Process • A plan clearly stated as a set of competencies • Systematic documentation • Examination and interpretation of quantitative and qualitative data • An adjustment of teaching/learning strategies to make identified improvements or changes • A focus on the student rather than the teacher

11. System-wide Institutional Program Assessment Course Assessment Classroom Assessment State-wide entry level assessment instruments College-wide surveys, Annual Assessment Report Licensure exam results, national tests, capstone course Pre-post test, simulations, projects Background knowledge probe, minute paper, muddies point Types and Levels of Assessment

12. Characteristics of Good Assessment Plans • Flows from college mission • Focus on the program rather than courses • Product of input/discussion by entire dept • Ongoing not periodic • Manageable • Uses multiple measures—direct & indirect—qualitative and quantitative • Students understand their role • Results used to improve

13. How effective assessment benefits departments • Faculty are clear about student learning objectives for their program area; they can teach more effectively • Rationale for curriculum design/course sequencing is more clear so advising and teaching are improved • Able to show evidence to outside constituents • Students learn better when we and they are clear about what they are expected to learn • Students learn better when they receive frequent specific feedback about their work or skills acquired • Faculty know more about student learning and can direct their teaching efforts accordingly

15. The Assessment Process

16. Learner outcomes • Provide direction for all instructional activity • Inform students about the intentions of the faculty • Are student-focused • Focus on learning resulting from the activity • Focus on important aspects of learning that are credible to the public

17. Learner outcomes (cont.) • Focus on skills and abilities central to the discipline • General enough to capture important learning but clear and specific enough to be measurable • Focus on aspects of learning that will endure but can be assessed in some form now

18. Relationship among outcomes • Institutional Outcome • Students will be able to speak and write effectively. • Program Outcome • Students can articulate principles and concepts of the discipline of nursing. • Course Outcome • Student can prepare a written summary and interpretation of a care plan.

19. Writing measurable objectives What is the student expected to know (Cognitive Learning), think (Affective Learning) , or be able to do (Skill Acquisition) at the end of a course, end of the year, etc.

20. Bloom’s Classification—Cognitive Skills • Knowledge—Count, Define, Recall • Comprehension—Compute, Explain, Summarize • Application—Apply, Solve, Use • Analysis—Compare, Diagram, Modify • Synthesis--Build, Produce, Revise • Evaluation—Conclude, Judge, Select

21. WHO? • Clearly, the audience is students. Do you intend the outcome to reflect all students? • Students who graduate? • Students in internships, coop experiences? • Students at the end of their first year? • Students who successfully complete a capstone project or experience?

22. WHAT? • What action do you expect? • It must be observable and measurable. • Use action words such as: list, name, construct, assemble. • What must the student do to convince you that they know, have learned, or appreciate the outcome you are expecting?

23. HOW? • Under what conditions is it to occur? • Upon completion of the course • At completion of the course/semester/program • In a simulated environment • Using a textbook for reference -- using no references

24. TO WHAT EXTENT? • What is the standard the student must achieve? • Is it a degree of acceptable performance – 80% accuracy, 8 of 10 established outcomes, comprehension level, etc. • Within a certain period of time? Two weeks, 60 minutes, etc.

25. Exercise • Review student learning outcomes • Relevant? Current? • Do they clearly state what students are expected to do? • Do they incorporate levels of Bloom’s taxonomy? • Includes standards? Expected success rate? • Write a student learning outcome that incorporates Who, What, How, and To What Extent? -- Share

26. Multiple means of measurement • Why don’t grades count as a measure of student learning? • Use measurements that take into account the different learning styles of students. • Do students have the ability to display their learning strengths—visual, auditory or kinesthetic through various methods of measurement? • The measurements should be both formative and summative.

27. Assessment Techniques • How can tests, presentations, and classroom assignments that you are already using be used to evaluate student learning in another way (besides a grade)? • Primary trait analysis • Rubrics • Scoring sheet • Concept mapping • Portfolio development

28. Measures of Student Learning • Direct indicators of learning • Pre and post testing • Capstone course or experience • Oral examinations • Portfolio assessments • Standardized national exams • Locally developed tests • Performances • Concept mapping

29. Measures of Student Learning • Indirect indicators of learning • Alumni surveys • Information gathered from students, parents • Graduation rates • Drop-out rates • Success of students in subsequent settings • Number and GPA of students who go on to get a Bachelor’s degree • Success of graduates in their field of study

30. Data Collection Methods • Paper and pencil testing • Computerized testing • Essays and writing samples • Portfolio collections of student work • Exit interviews • Surveys • Use of external evaluators • Logs and journals • Behavioral observations

31. Classroom Assessment Techniques • Classroom assessment techniques are simple assessments designed to assist you in determining student learning in specific instances • Background knowledge probe • Pre-post test • Minute paper • Focused listing • Concept mapping

32. Background Knowledge Probe • 1. What statements are true about leadership? ___Leaders are born, not made. ___Leadership exists only at the top. ___A good leader controls, directs, takes charge, corrects and monitors. ___Leaders support, coach and facilitate the work of others.

33. Minute Paper • What is the most important thing you learned today in this class? • What was the most powerful image in the reading selection for today? • What was the most convincing argument? • What important question remains unanswered? • What remains unclear?

34. Focused Listing • Use this technique to assess students’ understanding of a concept. • “You have two minutes to make a focused list of short words or phrases that you would use to describe or explain the elements of an effective treatment plan.”

35. Assessment Techniques • How do you use tests, presentations, and classroom assignments that you are already using and evaluate them another way? • Primary trait analysis • Rubrics • Scoring sheet • Portfolio development

36. Primary Trait Analysis • A method of constructing a common scoring format so that criteria can be compared and aggregated across courses or programs • Specific traits are identified and a specific score assigned • Results can be expressed as descriptions of skills being required

37. HOW TO CONSTRUCT A PRIMARY TRAIT SCORING SCALE • Choose a test or assignment that tests central goals/objectives of the course • Identify factors or “traits” that will count in the assessment. Use nouns: thesis, eye contact, voice inflection, etc. • For each trait, construct a 3-5 point scale. These are descriptive statements

38. Primary Trait Scale for Office Management • Objective: Apply technical skills necessary to perform spreadsheet applications in a hypothetical situation. • Trait: Define the spreadsheet layout in a logical format and decide the formulas and features to be used.

39. Scale

40. Primary Trait Analysis – English Dept QUESTIONYESNO • I know what plagiarism is. 99% 1% • I know how to avoid plagiarism 93% 2% • I know how to use parenthetical citation 84% 12% • I document paraphrases 93% 7% • I document summaries 85% 15% • I document quotations 97% 3% • I know how to cite electronic sources 84% 16% • I know how to create a Works Cited page 97% 3%

41. Instructor analysis TRAITYESNO Did the student… • Avoid plagiarism? 90% 10% • Use proper parenthetical or other citation? 58% 42% • Properly document paraphrases? 80% 20% • Properly document summaries? 80% 20% • Properly document quotations? 80% 20% • Properly document electronic sources? 55% 45% • Create a proper Works Cited page? 63% 37%

42. Constructing a Concept Map • Begin with a familiar segment of the text, lab activity or field activity—it acknowledges learner’s prior knowledge • Demonstrates what mastery looks like • Makes thinking visible—presents and integrates knowledge • A Focus Question clearly specifies the problem or issue the concept map should help to resolve – Ask the right question • Identify key concepts that apply (15-20)

43. Rubrics • Often designed for essays or other written project • Rubrics give detailed descriptions of what is required in each category • A score is assigned based on the category that best fits the written work

44. What is a Rubric? • Descriptive scoring scheme • Guides analysis of student’s work • Lays out specific expectations for an assignment • Provides expectations for what is acceptable and unacceptable work

45. Uses • Research papers • Lab reports • Portfolio evaluation • Group work • Oral presentations/Speeches • Demonstrations

46. Why use a Rubric? • Students know exactly what is expected of them • Cuts down on grading time • Provides fuller feedback to students—they know the areas of strength and areas they need to improve • Affects classroom preparation • Allows us to communicate our intentions and expectations more clearly • Encourages critical thinking

47. How do you know you need a rubric? • Carpal tunnel is setting in • Students complain they can’t read your handwriting • You can’t remember how you graded the first papers in the stack compared to the last ones • After laborious explanation, students still have questions

48. Basic parts of a Rubric • Task description • Scale • Dimensions of the assignment • Description of levels of performance

49. Task Description • Taken from the syllabus • Reminder of how the assignment was written as we grade • Have a reference for use later on

50. Scale—Levels of Achievement • Describes how well or poorly task was performed • Begin with three levels and expand up to five as you refine your assignment and expectations—can be weighted • Examples: • Exemplary, proficient, marginal, unacceptable • Advanced, intermediate, novice • Excellent, competent, needs work