University of Arkansas Faculty Senate Task Force on Grades

1. University of ArkansasFaculty Senate Task Force on Grades Preliminary Report April 19, 2005

2. Mandates for the Task Force • Does grade inflation at the U of A exist and is it a significant issue of faculty concern that may require policy change? • Are grades and any perceived or real grade inflation with time, or uneven grade distribution linked to uniform faculty evaluations (Purdue System)? • If there IS a relationship between Teaching Evaluations and Grading, … is it a significant problem that should be addressed? • What are Task Force Recommendations related to grades, grade definitions, etc., … that should be a faculty matter.

3. U of A not Unique! Academe Interested in Grades and Grading Practices • Duke University • Harvard • Dartmouth • Southwest Missouri State University • University of North Carolina • Princeton

4. Assignment of Grades • Philosophical Discussion (Faculty) • Why do we assign grades? • What does a grade represent? • Types • Criterion or Norm Referenced • Pragmatic Examination (Task Force) • What grades were assigned? • Are there patterns in the assignment of these grades that may represent a greater systemic problem?

5. Student Evaluation of Instruction • Philosophical Issues (Faculty) • Purpose of student evaluations? • Does the Purdue System provide the necessary information to evaluate instruction? • Differentiating a high rating from effective instruction and possible “inflation” of grades • Pragmatic Issues (Task Force) • Are There Patterns in Student Evaluations Associated with • Overall Grades Assigned • Type of Course • Faculty Rank

6. Increasing Grades? • Since mid-60s universities have seen undergraduate GPAs steadily increase • College Remedial Courses • Annual rate approaches 60% in Arkansas • 34% of Students in Arkansas Identified as “college ready” • Numerous theories exist and the issue is commonly referred to as “grade inflation”

7. Grade Inflation • Economic Description • Grades Increasing Over Time (Dow Jones) • Grade Inconsistent with Demonstrated Achievement • Student Assigned Grade of “A” with Limited Achievement • Does not Accurately Reflect Performance • Covering 9 versus 17 Chapters

8. The “Gold” Standard • The goal is to create a common standard to compare students and grades assigned • Use of standardized test information • ACT • CAAP “Rising Junior” Exams • GRE

9. Requested Information for Study • Grades Assigned and Student Evaluations (1992-2004) • Year • College • Department • Program • Section • Example of Demographic Variables • First Year Enrolled • Status (Transfer/Freshman) • High School GPA • Degree and Year Completed Note: 1992 – 2004 Represents the Time Frame Computerized Data Available

10. Federal Education Rights and Privacy ACT (FERPA) • Institutional Review Board • Provided Approval • Anonymous student and faculty identification numbers used • Additional Protection: If sample size was less than 10 observations an any analysis level, information was not reported.

11. Methodology • Checking the Data! • Initial Calculations • Computing student GPAs independently • Correlated 1.0 with University reported GPAs • Note: Computed with only fall and spring grades • Modifications for Analysis Purposes • Conversion of grades to whole values for specific analyses • Faculty Evaluations • Use of means in analysis of faculty evaluations

12. Research Design • Exploratory Data Analytic Techniques • Tukey (1978) • Graphing • Descriptive Statistics • Confidence Intervals • Trend Analysis • Layered Overview of Results • University -- College -- Department -- Program -- Course -- Section

13. Preliminary Results: UndergraduateU of A Grades from 1992 - 2004 2.95 2.76 Mean GPAs increased from 2.76 to 2.95 during 12 year period

14. College Undergraduate GPA Trends with U of A GPA Trends for 1993 - 2004 2004 GPAs EDUC= 3.52 AFLS= 3.14 ENGR= 3.06 ARSC= 2.82 ARCH= 2.82 WCOB= 2.77

15. University and College GPA Trends 1992 – 2004: Rescaled 3.52 Rosenthal (1973), studies on interpreting graphs 2.77 Rescaling on 0.0 to 4.0 grade scale may create a different impression

16. Student Evaluations and Grades Assigned: Is there a pattern? 86% of students expect grade of A or B Steady decline, then levels Based on the six Teaching Evaluation items all instructors must ask students! The correlation between expected grade and university core average was .29, suggesting only a moderate association

17. What Represents an Example of Effective versus Ineffective Grading • High or Low Grades? • Indicators of success? • Indicators of ineffective instruction? • Grade inflation? • Patterns! • Identify patterns in the data that demonstrate cause for concern or discussion

18. Demonstrate a Process for Assessing the Assignment of Grades • College of Education and Health Professions • Department of Educational Leadership, Counseling and Foundations • Educational Foundations Program • Graduate Degree Program Goal: To provide a method for evaluating the assignment of grades in a program area!

19. Graduate GPAs at the U of A 2004 GPAs EDUC= 3.77 ENGR= 3.70 ARSC= 3.66 AFLS= 3.65 WCOB= 3.55 Overall, graduate GPAs higher and have increased.

20. College of Education and Health Professions: Department Graduate GPAs Interesting! The GPAs by Dept. are becoming more variable

21. Educational Leadership, Counseling and FoundationProgram Area Graduate GPAs 1992 - 2004 3.89 However, not the complete picture! 3.48 Ed. Stat grades declining relative to rest of department

22. Educational Foundations: Evaluating our Grading Patterns1992 - 2004 Tenure track faculty assign grades much lower than adjuncts in Ed. Stats Volatility due to one course

23. Implications of Internal Analysis • We …. We the faculty in Educational Research and Policy Studies need to do a better job of overseeing/selecting our adjuncts. • We …. We need to individualize the issue of grades, expectations, and assignment of grades. • We …. We need to examine the content level and determine if expectations are commensurate with benchmark institutions. • Multiple Regression Example

24. Preliminary Conclusions • Grades are increasing! • Is this problematic? • Possible Explanations/Further Research • Higher entrance scores (ACT exams) • Transfer students • Transferring credits from other institutions (Community Colleges, Universities, etc.) • Academic expectations • Course taking patterns by students • Plus, many others ….

25. Preliminary Recommendations • Stair-Step Evaluation of Grades Information • Data Provided to Each College for Evaluations • Committees to examine information at each level and report -- College -- College Reports -- Department -- Department -- Program -- Program -- Course -- Course Evaluate -- Section  -- Section Ongoing Accountability

26. Next Steps • Final modifications of requested data received on March 18th . • Preliminary analyses completed, more in-depth study of issues during the next four months to understand the implications. • Comprehensive Report completed and submitted to Faculty Senate fall of 2005.