Download
1 / 70
700 likes | 795 Views
IDEA Trends at CNU. Deb Moore Director, Office of Assessment, Evaluation, & Accreditation 2011-2012 IDEA Taskforce. Internal and Benchmark Profiles.
E N D
IDEA Trends at CNU Deb MooreDirector, Office of Assessment, Evaluation, & Accreditation 2011-2012 IDEA Taskforce
IDEA Trends at CNU Internal and Benchmark Profiles • This presentation provides selected elements from 4 reports, 3 of which are internal profiles, and a 4th which displays trends for CNU with 3 benchmark comparisons. • The 3 internal profiles reflect the critical terms in which administration shifted largely from paper to online. • The external benchmark report provides 5 years of trend information. • One purpose of this presentation is to provide information. A second purpose is to provoke conversation among instructors, at multiple levels, about their intended outcomes and possible changes in strategies and methods to bring about learning improvements. • OAEA welcomes recommendations for additions and corrections to the content of this presentation; contact Deb Moore (deborah.moore1@cnu.edu).
IDEA Trends at CNU Information Sources • http://www.cnu.edu/assessment/idea/index.asp provides links to the full content of the following reports: • Institutional Summary Fall 2011 • Institutional Summary Spring 2011 • Institutional Summary Fall 2010 • IDEA Benchmarking for Learning Report* * 5 year trend report (2007-2011)
IDEA Trends at CNU A Look at IDEA for Three critical Terms Based on Institutional Group Summaries for Fall 2010, Spring 2011, & Fall 2011
IDEA Trends at CNU IDEA Administration During Key Terms • Fall 2010 - largely paper • Spring 2011 – mixed, more paper than online • Fall 2011 - largely online
IDEA Trends at CNU Instructors’ Reports on Course Emphases: Selected Pairings-Reading and Memorization
IDEA Trends at CNU Instructors’ Reports on Course Emphases: Selected Pairings-Writing and Oral Communication
IDEA Trends at CNU Instructors’ Reports on Course Emphases: Selected Pairings-Critical Thinking & Writing
IDEA Trends at CNU Instructors’ Reports on Course Emphases: Selected Pairings-Critical Thinking & Memorization
IDEA Trends at CNU Highlights • Remarkably similar profiles across terms • Overall response rates ranged from 75% to 81% • 1stterm in which administration was primarily online achieved a 75% response rate • Transition from paper to online (fall 2010 to fall 2011) does not show major differences in profiles • CNU faculty focus on 3-4 outcomes as essential/important • IDEA scores for CNU faculty are in the average range on each of the 4 primary indices of teaching effectiveness • Over the last 3 terms, a significant increase on several objectives has been observed: application of course material, oral and written communication skills, & analysis and critical thinking skills (objectives 3, 8, & 11, respectively)
IDEA Trends at CNU 5 Year trends with external benchmarks Based on IDEA Benchmarking for Learning Report
IDEA Trends at CNU General Description • Uses data from 2007 to 2011 • Includes 3 comparison (benchmark) groups • Captures segments of scales (Instructors’ ratings of Essential and Important; Students’ ratings of Exceptional and Substantial) • Unit of analysis for this report is the individual rather than class (as done for group summary reports)
IDEA Trends at CNU Comparison Groups * Peer group is based on 10 institutions identified by the Provost staff and Deans
IDEA Trends at CNU Report Content • Overall Progress on Learning (S) • Frequency of Learning Objective Selection (F) • Progress on Learning (S) • Teaching Method Emphasis (F) • Student Characteristics (F) • Summary Ratings of Effectiveness (S) • Faculty Ratings of Other Impacts on Learning (F) Appendices • Learning Objective Selection and Progress (F & S) • Teaching Methods Emphasis Over Time (F) This presentation does not follow the order of the report and it does not include all the material found in the 5 year trend report.
IDEA Trends at CNU Judging What Is Important • Differences greater than 10% are important and uncommon • Differences between 5 and 10% are likely important • Differences less than 5% are not likely to be meaningful
IDEA Trends at CNU Response Rates CNU student participation is similar to that of each comparison group CNU=79% Peer=77% Carnegie=79% National=75%
IDEA Trends at CNU Key Questions to Explore • How do CNU’s results compare to peer institutions? • Have there been changes over time? • Are results for certain levels and purposes different from the overall results? • When comparing CNU’s results to the comparison groups’, is the pattern similar regardless of course level and purpose?
IDEA Trends at CNU Overall Progress on Learning
IDEA Trends at CNU Students’ Perceptions
IDEA Trends at CNU Trends: Student Perceptions • Small, steady positive trend over last 5 years • CNU clusters with the other comparison groups • Although trends for CNU are higher than for other groups, these are not meaningful differences • Of the 5 contexts identified on the FIF, the strongest positive perceptions at CNU were for progress on objectives in the graduate education context; this represents a narrow, yet important, segment of CNU programs (see next slide)
IDEA Trends at CNU Graduate Students
IDEA Trends at CNU Frequency of Objectives selected
IDEA Trends at CNU Trends: Instructors’ Responses • CNU selected fewer learning objectives per class (3.7) than other comparison groups (ranged from 5.0 to 5.5) • At CNU, objectives 1, 2, & 3 are most frequently selected • Overall, CNU trend pattern is similar to others but generally with a gap between CNU and comparison groups; CNU’s instructors select objectives less frequently than do instructors of other groups (10 of 12) • The gap between trends lines is associated with the smaller number of objectives routinely selected by CNU faculty in contrast to the average number by benchmark groups (confirmed interpretation with IDEA)
IDEA Trends at CNU Trends: Students’ Responses • Positive progress reported on objectives emphasized by faculty • Overall, CNU trend pattern is similar to others but some gaps between CNU and comparison groups • Gaps vary with respect to different comparison groups by objective • Same or higher than comparison groups when gaps are observed (Objectives 5, 6, 7, 8, & 11)
IDEA Trends at CNU Instructional Objectives Selected by Instructors Instructors’ Intentions/ focus Students’ Self-Reported Progress on Learning
IDEA Trends at CNU Organized by Highest FrequencyReported by CNU Instructors
IDEA Trends at CNU Organized by Highest FrequencyReported by CNU Students
IDEA Trends at CNU Students’ Perceptions of Teaching Methods
IDEA Trends at CNU Teaching Methods and Styles Reported by Students(Diagnostic Forms Only) Fostering Student Collaboration Encouraging Student Involvement
IDEA Trends at CNU Teaching Style Scales Fostering Student Collaboration Encouraging Student Involvement 9. Encouraged students to use multiple resources (e.g. data banks, library holdings, outside experts) to improve understanding 11. Related course material to real life situations 14. Involved students in “hands-on” projects such as research, case studies, or “real life” activities 19. Gave projects, tests, or assignments that required original or creative thinking 5. Formed “teams” or “discussion groups” to facilitate learning 16. Asked students to share ideas and experiences with others whose backgrounds and viewpoints differ from their own 18. Asked students to help each other understand ideas or concepts
IDEA Trends at CNU Fostering Student Collaboration Trend line for CNU is fairly steady over 5 years.
IDEA Trends at CNU Encouraging Student Involvement Trend line for CNU is fairly steady over 5 years.
