Micro-credentialing of English Learner Teaching Skills Through Digital Badges

1. Micro-credentialing of English Learner Teaching Skills Through Digital Badges University of Central Florida Atlanta, Georgia TESOL 2019

2. Teacher Education Research & Curriculum Development Project U. S. Department of Education • Funded by the Office of English Language Acquisition, US Department of Education • National Professional Development Grants (2016) • Currently in Year 3 of a 5 year grant • Initial data is being collected

3. Project Participants Elementary Education Cohorts Regional Campuses Study Began in Fall 2017 and Ends Fall 2020 Presentation focuses on badging data collection

4. Digital Badging Assessment Process Complete video-based modules of 10 key instructional skills for supporting elementary ELs at varying WIDA levels in 8 teacher preparation courses (webcourses.ucf.edu) Practice skills through micro-teaching or TeachLive Demonstrate instructional skill before an expert panel and obtain micro-credential in the form of Digital Badges

5. 5 5 Ten Targeted Instructional Skills

6. 6 6 Digital Badges Credly

7. Regional Campuses One Plus Model of EL Infusion 10 Video Modules Main Campus One Plus Model of EL Infusion Skill Coaching through TeachLivE Regional Cohorts Skill Coaching through Micro-Teaching Regional Cohorts Assessment of the 10 Skills & Micro-Credentialing with Digital Badges

8. Process of Inclusion Develop the 10 video modules & practice activities/coaching protocols • Course Shepherds Identified Potential Assignments for Modules • MELTS Team Distributed Video Modules and Coaching/Badging Rubrics for Review • MELTS Team Completed Video Module Revisions based on feedback received (ongoing) • Video Modules were uploaded into Webcourses as stand-alone supplementary modules Elementary Faculty Course Shepherds Grant Team Regional Coordinators

9. Sample Coaching Rubric – Skill One NOTES:_______________________________________________________________________________

10. Sample Badging Rubric – Skill One

11. Ongoing Research Protocols Interviews and focus groups with: Teacher candidates Course instructors Observation and survey instruments developed and used for: Micro-credentialing skill assessments TeachLivE and micro-teaching activities Teacher Work Sample Sample of teacher candidates during classroom instruction in Internship 2 Transcribe and analyze: Video-recorded samples of skill assessments TeachLivE and micro-teaching activities

12. Extended Research ANALYZE the micro-teaching and TeachLivE groups’ EFFECTS ON THEIR ENGLISH LEARNERS’ GAINS ON UNIT TESTS compared to the gains of non-English learners COMPARE both experimental groups’ student gains and instructional practices WITH A CONTROL GROUP of teacher candidates

13. Exploratory Research on Digital Badges- Cohort 1 Research Questions (1) What is the success rate of teacher candidates in completing the necessary requirements of the MELTS badge system to be awarded a digital badge for English learner teaching skills? (2) Do teacher candidates perceive that the skill practice sessions prepared them to successfully pass the MELTS badging assessment and serve EL students in the future?

14. Methodology Participants Undergraduate TCs (K-6) in fall 2017 completing five MELTS badges (Skills 1, 2, 5, 6 and 10). N=151 Analysis The study collected quantitative data on (a) participant completion and (b) participant perception of the skill practice collected by the survey. Data were analyzed by the authors using descriptive statistics (frequency and percentages). No inferential statistics were used.

15. RQ 2 Survey Twenty-six participants voluntarily completed a 14-item survey regarding their perception of the practice session including both quantitative and open-ended questions The survey was designed and analyzed by the university Program Evaluation and Educational Research (PEER). A total of 33 participants completed the survey.

16. RQ One Results- Badging Completion • Cross Tabulation of Badges Earned by Participants by Skill

17. RQ One Results- Badging Completion Cross Tabulation of Participant Successful Completion of the MELTS DBS Components by Skill

18. RQ 2 Results- Practice Session Surveys In aggregate, 92% of the respondents rated the coaching sessions in both environments as effective. Among those who completed micro-teaching, all but one (94%) agreed it was effective, while all but one (88%) rated the TeachLive session effective. The majority agreed that they felt more comfortable working with ELs, managing their behavior, and supporting their needs. They enjoyed the experience and agreed it should continue in the program.

19. Results: Discussion Success of the digital badging process. Because digital badges are earned with varying levels of evaluation (Hills & Hughes, 2016), we wished to explore whether this rather rigorous assessment system would be effective. Results were positive, but with one caveat Skill practice session perceptions. Overall, TCs expressed satisfaction with the skill practice sessions, in both low-tech and high-tech formats, with the high-tech TeachLivE environment perceived slightly better.

20. Results: Recommendations Logistics: As noted by other researchers (i.e. Abramovich, Schunn, & Higashi, 2013; Casilli & Hickey, 2016), addressing logistics and value within any DBS is key to ensuring successful participation. Motivation: A core element of DBs is learner autonomy, so TCs were allowed to choose whether to participate in the MELTS DBS or not. It is clear that the value of Digital Badges needs to be discussed and promoted with all participants. Impact: More research is needed to determine the impact of digital badging on learning and assessment (Devedžić & Jovanović, 2015), particularly on teacher skill development and how that translates into long-term, classroom instructional techniques (Gamrat, Zimmerman, Dudek, & Peck, 2014; Gibson, et al., 2015).

21. References Abramovich, S., Schunn, C., & Higashi, R. M. (2013). Are badges useful in education? It depends upon the type of badge and expertise of learner. Educational Technology Research and Development, 61(2), 217-232. Casilli, C., & Hickey. D. (2016). Transcending conventional credentialing and assessment paradigms with information-rich digital badges, The Information Society, 32(2), 117-129. Devedžić, V., & Jovanović, J. (2015). Developing open badges: a comprehensive approach. Educational Technology Research and Development, 63(4), 603-620. Gamrat, C., Zimmerman, H. T., Dudek, J., & Peck, K. (2014). Personalized workplace learning: An exploratory study on digital badging within a teacher professional development program. British journal of educational technology, 45(6), 1136-1148. Gibson, D., Ostashewski, N., Flintoff, K., Grant, S., & Knight, E. (2015). Digital badges in education. Education and Information Technologies, 20(2), 403-410. Hills, L., & Hughes, J. (2016). Assessment worlds colliding? Negotiating between discourses of assessment on an online open course. Open Learning, 31(2), 108–115.

22. Grant Team Members This study was funded through a grant provided by the Federal Department of Education, Office of English Language Acquisition, Grant ID: 14236011 • The MELTS grant research team consists of: • Joyce Nutta, Principle Investigator • Florin Mihai, Co-Principle Investigator • Kerry Purmensky, Co-Principle Investigator • Leslie Davis, Project Manager • Laura Monroe, Post-Doc Researcher • Alex Davis, Graduate Research Assistant • Ying Xiong, Graduate Research Assistant • Shizhong Zhang, Graduate Research Assistant • Nirmal Ghimire, Graduate Research Assistant

23. Grant Team Members Find our PPT on www.projectmelts.org under For Researchers