Technology Integration in K-12 Education: A Performance Analysis using the Delphi Methodology

1. COVER

2. Presentation Outline I. Introduction II. Performance Analysis III. Delphi Methodology IV. Data Analysis V. Discussion VI. Next Steps VII. Conclusion

3. Foundation History Founded by Lori Remley Mody, formerly of SIGNAL Corporation (now General Dynamics) McLean, Virginia Mission: “to build Communities of Practice that support the advancement of K-12 education through technology.”

4. Rational • Technology, in its many forms has become a powerful tool to enhance curriculum and instruction (ISTE, 1999) • Teachers lack the support and resources to effectively use technology in the classroom • NCLB mandates accountability for teachers, administrators and school districts (U.S. Department of Education) • Increased pressure for authentic learning environments that teach higher order thinking skills

5. GMU-Win Win Partnership • Action-based learning approach to instructional design that bridges theory to practice • Expertise in instructional systems design and instructional design technology • Expertise in building online learning environments

6. ADDIE MODEL • Analysis • Design • Development • Implementation • Evaluation

7. ADDIEMODELAn iterative process… • Analysis phase has two parts: • Performance analysis • Needs analysis

8. Analysis

9. Performance Analysis A performance analysis involves a partnership with the client to help define and achieve their goals. This involves identifying: • Several Perspectives • Drivers • Barriers • Solution system

10. Performance Analysis Sources: Through interviews, surveys, focus groups • Executives • Experts in the context domain • Colleagues • People in “the trenches” Additional sources: • Research & Studies • Records & Tests • Course Materials • Work Products

11. Performance AnalysisPurpose Determining the investigative tool for a Performance Analysisdepends on whether your goal is: • An opportunity • A problem • A development of a group of people • A strategic plan

12. WWSF Performance Analysis conducting research into how technology is currently being integrated into high schools, after-school programs, and CoPs Sources: • Discussions between the client and team members • Printed material supplied by the client • Research from Journals, CoPs, Case Studies • Consultation with expert faculty at GMU • Panelists • Online research and literature

13. Project Vision Our vision is to enhance the experience of teachers and students through the integration of technology. Project Mission/Goals To achieve our vision, we will identify the best practices for the integration of technology in high school in order to support a community of practice.

14. Project Stakeholders • Researchers at universities and nonprofits • Policy Makers at the national, state and district levels • Administrators at district and site levels, including Principals and Program Directors • Teacher leaders, school technology coordinators, and librarians

15. Project Goals • Research and identify national best practices for using technology in HS • Identify opportunities/problems in educational technology • Establish network of panelists in educational technology • Establish links to community-based education and communities of practice

16. Project Drivers "The nature of the drivers influences the nature of the solution system that is proposed." (Rossett, 1999) Influences are often categorized as drivers. Therefore, it is important to fully understand the drivers that may be influencing the successful integration of technology in high schools.

17. Project Drivers • The need to improve the integration of technology in the high school environment • Research on Best Practices • Desire to make a change • Team and client commitment to vision • Collaboration with the client

18. Project Barriers "...any constraints that are perceived to be getting in the way of performance and accomplishment"  Rossett (1999).

19. Project Barriers • Inadequate research-based literature and data • Lack of access to Experts • Inadequate experts participation/feedback

20. Project Recommendations • Conduct a Needs Analysis using the Delphi methodology as the needs assessment tool to determine the best practices of technology in high school, with special attention to communities of practice and after-school programs • Research and present 3 papers on areas concerning the integration of technology to enhance high school student achievement within the context of: • High-Tech High schools, • After-school and Community Technology Programs • Communities of Practice.

21. Needs Analysis

22. The Delphi Methodology Conception: 1950s, “Project Delphi” Defining characteristics and purpose: • Gathers subjective insights from experts from diverse backgrounds in terms of experience and expertise • Allows experts to coalesce around a complex issue and work towards some degree of consensus • Facilitates group “discussion”, anonymously if done online • Useful when it is necessary to involve more individuals than can be physically grouped together for lack of time, proximity, or other resources

23. Delphi Process • Identify the purpose: This is the over-arching issue the study will explore. • Create the categories of expertise: Take into consideration all the stakeholders. • Articulate the criteria for selection and fill the panels with experts • Round One: This round begins by developing an open-ended, unbiased question based on the overarching issue. The question, or questions, are then delivered to the experts to solicit their input.

24. Delphi Process (cont.) • Analysis of responses: This results in a list of items distilled from the input. • Round Two: This round asks panelists to rate the individual items distilled from the first round in terms of their importance. • Calculation of panel means: This step reveals where there is already consensus and where there is not. The items of non-consensus then move to the next round. • Round Three: This round offers panelists an opportunity to re-think their initial rating in light of the feedbackfrom all panels and works to achieve a greater degree of consensus among panels.

25. Panels

26. Panelist selection criteria

27. Developing Question for Round One The purpose of this question is to gather the expert’s input regarding the overall issue of the study. Question should be: • Open-ended • High-level • Focused, but not biased Once the question has been established, and coded, it is sent out to the panelists. The coding is used to protect the anonymity of the panelists, and the format looks like this: R#P#I#

28. Round One

29. Round Two Purpose The purpose of this round is to determine where consensus already exists so that the research can focus on where it doesn’t. The format changes from open-ended and qualitative to focused and quantitative. The panelists are asked to rate the 34 items distilled from Round One in terms of their importance.

30. Round Two When planning for the role of technology in support of teaching and learning in high schools, how would you rate the importance of the following considerations and practices? 1. Item here O Absolutely Essential   O Very Important   O Important   O Marginally Important   O Not Important Comments:

31. Round Three Purpose The purpose of this round is to work towards a greater degree of consensus. This is done by presenting the panelists with the feedback from all of the panels. Allows panelists to rethink their initial rating in light of the new information they have. This round ultimately exposes where there is and is not consensus among the experts.

32. Round Three (Note: M = Mean) Teacher Leaders/*************************/*M***************/**********************/********************/Absolutely Essential          Very Important                 Important                           Marginally Important         Not Important Administrators /*************************/*M***************/**********************/********************/Absolutely Essential          Very Important                 Important                           Marginally Important         Not Important Researchers /*************************/*M***************/**********************/********************/Absolutely Essential          Very Important                 Important                           Marginally Important         Not Important Policy Makers /*************************/*M***************/**********************/********************/Absolutely Essential          Very Important                 Important                           Marginally Important         Not Important (Comments of all panelists were included here) Your rating and comments: O Absolutely Essential   O Very Important   O Important   O Marginally Important   O Not Important Comments: 

33. Participation * *Round three saw an increase in participation because only those who had participates in one or both of the first two rounds were asked to participate in this final round.

34. Results

35. Round One: Respondents There were 41 Respondents

36. Round One: Data Analysis • 184 Separate considerations and practices • Data Analysis • Identified content overlap • Combined similar ideas • Discussed intricacies of statement focus • 34 Considerations identified for Round Two • Distillation of the original considerations

37. Very Important • Absolutely Essential • Important • Marginally Important • Not Important Round Two: 34 Considerations Having access to technology in the classroom to support teaching and learning

38. Round Two: Respondents There were 49 Respondents

39. Round Two: Data Analysis • Data Analysis • Calculated Mean within panel • Calculated Mean across panels for each question • Computed Mean difference across panels for each question • Identified consensus/non-consensus items • Used .9 as threshold • 30 items reached consensus • 4 non-consensus items identified for Round Three for more input

40. Round Two: Ratings 5 Absolutely Essential  4 Very Important   3 Important   2 Marginally Important   1 Not Important

41. Round Three: Goal / 4 Questions When planning for the role of technology in support of teaching and learning in high schools, how would you rate the importance of the following considerations and practices? (Items: 3, 15, 19, 33) Ex: 19. Requiring colleges and universities to incorporate effective technology practices and methods into their teacher education curriculum and programs     (Note: M = Mean) Teacher Leaders/*****************/*M***************/*****************/********************/_________ Absolutely Essential           Very Important                 Important                     Marginally Important         Not Important

42. Round Three: Respondents There were 41 Respondents

43. Round Three: Data Analysis • Data Analysis • Calculated Means • Consensus reached on 3 of 4 items • Item 33 did not achieve consensus in round 2 or 3 • Non-consensus still rated > Important

44. Round Three: Ratings

45. Discussion of Results • Consensus overall • Despite the Diversity of perspectives • Teachers and researchers group together • Administrators and policy makers group together • Clusters formed • Data fell into six theme areas • School Culture • Community Connection and Support • Professional Development • Access • Research and Policy • Curriculum

46. Discussion (cont.) • Consensus reached on 33 items • Non-consensus on one item • Item 33 (safe school environment) • mean rated important

47. 5 Highest Rated Discussion (concluded) • Very important to Absolutely essential • “Context is everything” • The Mott Foundation

48. Theme Areas

49. Theme Areas

50. Community Connection and SupportObservations • Item 11, with a “very important” rating, has Teacher Leaders rating it the highest in the category and gave comments strongly supporting • Learn through sharing • Learn from peers • Item 14,with the highest rating in the category, indicated that • all panels rated this “very important” • Administrators gave it the highest rating Parental support and involvement through effective communication and outreach 14.