Center for Educational and Instructional Technology Research (CEITR)

1. Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review Center for Educational and Instructional Technology Research (CEITR) University of Phoenix School of Advanced Studies

2. Presented by:Dr. Mansureh Kebritchi, Research Directormansureh.kebritchi@phoenix.eduDr. Sandra G. Nunn, Research Fellow / Facultysandynunn@email.phoenix.edu

3. Introduction • Use of learning analytics (LA) in education • Key issue: How can we better identify methods, benefits, and challenges in LA for higher education? • This study reveals: • LA methods, benefits, and challenges through a systematic literature review • How these components affect educators and students in higher education

4. Overview • Comprehensive literature review • Cooper (1988) method used to synthesize the literature • Formulate problem, collect data, evaluate data, analyze data, organize/present results • Focused on articles regarding: LA methods, benefits, and challenges • Revealed insights relative to how these three components affect educators and students in higher education

5. Problem • Embracing LA in evaluating data in higher education diverts educators’ attention from identifying methods, benefits, and challenges • Few studies have synthesized prior studies or provided combined overview of LA in higher education • Need for further clarification of three components so educators can better apply LA in higher education.

6. Research Questions RQ1: What are the methods for conducting learning analytics in education? RQ2: What are the benefits of using learning analytics in education? RQ3: What are the challenges of using learning analytics in education?

7. Purpose LA uses predictive models that can help improve educational decision making / performance (Campbell, De Blois, & Oblinger, 2007) Increases learner awareness to help improve the educational process (West, 2012) Tracks and predicts learner performance and potential issues such as students at risk (EDUCAUSE, 2010; Johnson, Smith, Willis, Levine, & Haywood, 2011)

8. Significance of Study LA is an emerging field of education; therefore, stakeholders need to better understand LA methods and application in higher education (Scheffel, Drachsler, Stoyanov et al., 2014) Providing an overview is critical to understanding LA and its applications in higher education

9. Method • Literature Review Method to Synthesize the Literature (Cooper, 1988) • Formulate the problem • Collect data • Evaluate the appropriateness of the data • Analyze and interpret relevant data • Organize and present the results

10. Literature Review • Focused on peer-reviewed journals since 2000 to identify learning analytics methods, benefits, and challenges • Keywords included: • Learning analytics and methods • Learning analytics and benefits • Learning analytics and challenges • Data mining and education • Learning analytics and education • Learning analytics

11. Results • Methods • Prediction • Clustering • Relationship mining • Discovery with models • Separation of data for use in the process of human judgment (Baker, 2010; Baker & Yacef, 2009; Romero & Ventura, 2010)

12. Results • Benefits • Identification of target courses • Curriculum improvement • Student learning outcomes • Personalized learning • Improved instructor performance • Benefits to the research community

13. Results • Challenges • Data tracking • Data collection • Data analysis • Learning environment optimization • Emerging technology • Ethical issues

14. Implications • As a result of greater availability and access to data, LA will increase understanding of patterns of: • Learner behavior • Networks • Interactions

15. Conclusion • Schools must recognize importance of a data-driven approach to education • Use of performance systems allows for: • Better decision-making • Identification of trends and problematic issues • Ability to allocate resources more efficiently • Using LA ensures greater outcomes to improve quality of teaching and learning

16. References Baker, R. (2010). Data mining for education. International Encylopedia of Education, 7, 112-118. Baker, R., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–16. Campbell, J. P., De Blois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new tool for a new era. Educause Review, 42(4), 40-57. Retrieved from EDUCAUSE website located at: http://www.educause.edu/ero/article/academic-analytics-new-tool-new-era Cooper, H. (1988). The structure of knowledge synthesis: A taxonomy of literature reviews. Knowledge in Society, 1, 104-126. EDUCAUSE. (2010). Next generation learning challenges: Learner analytics premises. EDUCAUSE Publications. Retrieved from EDUCAUSE website located at: http://www.educause.edu/Resources/NextGenerationLearningChalleng/215028 Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K., (2011). The horizon report: 2011 edition (Report). Retrieved from https://net.educause.edu/ir/library/pdf/HR2011.pdf Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 40(6), 601-618. Scheffel, M., Drachsler, H., Stoyanov S., & Specht, M. (2014). Quality indicators for learning analytics. Educational Technology & Society, 17(4), 117–132.

17. References West, D. M. (2012, September). Data mining, data analytics, and web dashboards. Governance Studies at Brookings, 1-10. Retrieved from Brookings.edu website at: http://www.brookings.edu/~/media/research/files/papers/2012/9/04%20education%20technology%20west/04%20education%20technology%20west.pdf