Scaling

1. Scaling February 6 Melissa Koch & Torie Gorges SRI International Ball, D. L., & Cohen, D. K. (in preparation). Instructional improvement and the problem of scale.Unpublished manuscript.Blumenfeld, P., Fishman, B., Krajcik, J., & Marx, R. W. (2000). Creating useable innovations in systemic reform: Scaling up technology-embedded project-based science in urban schools. Educational Psychologist, 35(3), 149-164.Cobern, C. E. (2002). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12.Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12.Confrey, J., Lemke, J., Marshall, J., & Sabelli, N. (2001). Models of Implementation Research within Science and Mathematics Instruction in Urban Schools.Elmore, R. F. (1996). Getting to scale with good educational practice. Harvard Educational Review, 66(1).Ferrio, T. (2004). What year did the graphing calculator get to scale? (email correspondance).Fullan, M. (2000). The return of large scale reform.Fullan, M., & Earl, L. (2002). Large scale reform. Journal of Educational Change, 3, 1-5.Horne, M., Cheeseman, J., Clarke, D., Gronn, D., & McDonough, A. Professional development and effective teaching: Developing students mathematical thinking in the early years through enhancing teachers' knowledge.Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., et al. (2000). Manual for the Patterns of Adaptive Learning Scales (PALS). Ann Arbor, MI: University of Michigan.Oliver, R., & McLoughlin, C. (2003). Pedagogical designs for scaleable and sustainable online learning. In A. Littlejohn & S. Buckingham Shum (Eds.), Reusing online resources: Special issue of the Journal of Interactive Media in Education.Roschelle, J., Kaput, J., Stroup, W., & Kahn, T. (1998). Scaleable integration of educational software: Exploring the promise of component architectures. Retrieved January 8, 2003, 2003, from http://www-jime.open.ac.uk/98/6/Roschelle, J., Tatar, D., & Kaput, J. (in press). Getting to scale with innovations that deeply restructure how students come to know mathematics. In A. E. Kelly, R. Lesh & J. Y. Baek (Eds.), Handbook of Innovative Design Research in Science, Technology, Engineering, Mathematics Education. Hillsdale, NJ: Erlbaum.Ryser, G. R., Beeler, J. E., & McKenzie, C. M. (1995). Effects of a computer-supported intentional learning environment (CSILE) on students' self-concept, self-regulatory behavior, and critical thinking ability. Journal of Educational Computing Research, 13(4), 375-385.Schneider, B., & McDonald, S.-K. (Eds.). (2007). Scale-up in Education: Issues in Practice (Vol. 2). New York: Rowman & Littlefield Publishers, Inc.Schoenfeld, A. H. (2002). Making mathematics work for all children: issues of standards, testing, and equity. Educational Researcher, 21(1), 13-25.Shute, V. J., & Glaser, R. (1990). A large-scale evaluation of an intelligent discovery world: Smithtown. Interactive Learning Environments, 1, 51-76, 1(eractive Learning Environments, 1, 51-76), 51-76.Songer, N. B. (2007). Rethinking sustainability of curricular innovations: Notes from urban Detroit. In B. Schneider & S.-K. McDonald (Eds.), Scale-up in education: Issues in practice (Vol. 2, pp. 165-182). New York: Rowman & Littlefield Publishers, Inc.Thompson, D., & Senk, S. (2001). The Effects of Curriculum on Achievement in Second-Year Algebra: The Example of the University of Chicago School Mathematics Project.Torgerson, C. (2001). The need for randomised controlled trials in educational research. British Journal of Educational Studies, 49(3), 316-328. Ball, D. L., & Cohen, D. K. (in preparation). Instructional improvement and the problem of scale.Unpublished manuscript.Blumenfeld, P., Fishman, B., Krajcik, J., & Marx, R. W. (2000). Creating useable innovations in systemic reform: Scaling up technology-embedded project-based science in urban schools. Educational Psychologist, 35(3), 149-164.Cobern, C. E. (2002). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12.Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12.Confrey, J., Lemke, J., Marshall, J., & Sabelli, N. (2001). Models of Implementation Research within Science and Mathematics Instruction in Urban Schools.Elmore, R. F. (1996). Getting to scale with good educational practice. Harvard Educational Review, 66(1).Ferrio, T. (2004). What year did the graphing calculator get to scale? (email correspondance).Fullan, M. (2000). The return of large scale reform.Fullan, M., & Earl, L. (2002). Large scale reform. Journal of Educational Change, 3, 1-5.Horne, M., Cheeseman, J., Clarke, D., Gronn, D., & McDonough, A. Professional development and effective teaching: Developing students mathematical thinking in the early years through enhancing teachers' knowledge.Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., et al. (2000). Manual for the Patterns of Adaptive Learning Scales (PALS). Ann Arbor, MI: University of Michigan.Oliver, R., & McLoughlin, C. (2003). Pedagogical designs for scaleable and sustainable online learning. In A. Littlejohn & S. Buckingham Shum (Eds.), Reusing online resources: Special issue of the Journal of Interactive Media in Education.Roschelle, J., Kaput, J., Stroup, W., & Kahn, T. (1998). Scaleable integration of educational software: Exploring the promise of component architectures. Retrieved January 8, 2003, 2003, from http://www-jime.open.ac.uk/98/6/Roschelle, J., Tatar, D., & Kaput, J. (in press). Getting to scale with innovations that deeply restructure how students come to know mathematics. In A. E. Kelly, R. Lesh & J. Y. Baek (Eds.), Handbook of Innovative Design Research in Science, Technology, Engineering, Mathematics Education. Hillsdale, NJ: Erlbaum.Ryser, G. R., Beeler, J. E., & McKenzie, C. M. (1995). Effects of a computer-supported intentional learning environment (CSILE) on students' self-concept, self-regulatory behavior, and critical thinking ability. Journal of Educational Computing Research, 13(4), 375-385.Schneider, B., & McDonald, S.-K. (Eds.). (2007). Scale-up in Education: Issues in Practice (Vol. 2). New York: Rowman & Littlefield Publishers, Inc.Schoenfeld, A. H. (2002). Making mathematics work for all children: issues of standards, testing, and equity. Educational Researcher, 21(1), 13-25.Shute, V. J., & Glaser, R. (1990). A large-scale evaluation of an intelligent discovery world: Smithtown. Interactive Learning Environments, 1, 51-76, 1(eractive Learning Environments, 1, 51-76), 51-76.Songer, N. B. (2007). Rethinking sustainability of curricular innovations: Notes from urban Detroit. In B. Schneider & S.-K. McDonald (Eds.), Scale-up in education: Issues in practice (Vol. 2, pp. 165-182). New York: Rowman & Littlefield Publishers, Inc.Thompson, D., & Senk, S. (2001). The Effects of Curriculum on Achievement in Second-Year Algebra: The Example of the University of Chicago School Mathematics Project.Torgerson, C. (2001). The need for randomised controlled trials in educational research. British Journal of Educational Studies, 49(3), 316-328.

2. Piloting Build IT Implemented Build IT 3 years at Girls Incorporated of Alameda County, CA Build IT�s goals are to encourage middle school girls to Explore and pursue IT careers Use technology to strengthen and build their technology fluency Take high school algebra and geometry courses in preparation for postsecondary STEM education and/or IT careers In addition to these learning goals for the girls, Build IT aims to enhance GIAC�s staff capacity to offer IT fluency programming.

3. Key Elements of Build IT Problem-based curriculum that uses the Understanding by Design approach Embedded performance tasks for evaluating technology fluency Family Tech Nights Professional development materials for staff Guides for involving IT professionals Evaluation instruments for measuring girls� interests and understandings

4. Successes of the Build IT Pilot Girls image of IT careers as solitary and boring have changed significantly to collaborative, fun, and intellectually stimulating Girls have increased their technology skills and conceptual knowledge Girls expressed more interest in mathematics and computer science courses Staff have developed greater IT knowledge and skills

5. Piloting the Scaling of Build IT 11 Girls Inc. program sites implement Build IT over two years (2008-2010) 6 Girls Inc. affiliates (8 new program sites) applied and were accepted to participate Continued implementation at the Girls Inc. of Alameda affiliate (3 program sites) Partners Professional development Ongoing support beyond funding Research questions and evaluation approach Why pilot scaling? SRI�s experience with program and technologies and Girls Inc.�s approach as well. It is useful for development research to be conducted with a purposive sample of prospective users who represent extremes with respect to background variables that are likely to affect implementation . Initial pilot tests benefit from such diversity by making it more likely that those tests will reveal similarities and contradictions among different actors� perspectives on the cooperative activities in which they engage Why pilot scaling? SRI�s experience with program and technologies and Girls Inc.�s approach as well. It is useful for development research to be conducted with a purposive sample of prospective users who represent extremes with respect to background variables that are likely to affect implementation . Initial pilot tests benefit from such diversity by making it more likely that those tests will reveal similarities and contradictions among different actors� perspectives on the cooperative activities in which they engage

6. Eight New Program Sites Located in the Northeast U.S. and Canada to aid professional development and evaluation: Concord, New Hampshire Nashua, New Hampshire Holyoke, Massachusetts Lowell, Massachusetts Philadelphia, Pennsylvania Hagerstown, Maryland York, Canada (2 program sites) Selected based on their ability to support the program and for their diversity as a group Small stipend for participating ($10,000 per program site over two years.)

7. Criteria for Participation All program sites have High-speed Internet access and one computer for every two to three girls. Resources to implement at least 60 hours of the curriculum during the school year plus the 2-week summer program. Sites will consider implementing all 240 hours of the curriculum. Staff who are willing to learn technology and design through participation in Build IT professional development and have a commitment to implementing the Build IT curriculum. Implemented Girls Inc. Operation SMART? (Science, Math, and Relevant Technology) for a minimum of 1 year.

8. Diversity Among Program Sites Rural vs. urban School vs. center Demographics (range from 6% to 98% minority; more than half have at risk populations) All of the curriculum vs parts of the curriculum Time of year and duration (concentrated weeks vs. after school)

9. Partners Girls Incorporated of Alameda County Resource for program sites implementing Build IT Lead PD Girls Incorporated�s national office Ongoing PD Contact and resource support for affiliates SRI International Initial�PD to affiliates staff. Train-the-trainer model. Curriculum completion and support Formative evaluation Development of Program Adaptation Toolkit Funded by The Noyce Foundation. We have also requested a supplement to our current ITEST grant.

10. Professional Development Initial face-to-face two-day PD Eight web casts; 4 per year Online community (Tapped In) for leaders and participants to help each other; moderated by Girls Inc. and SRI. Opportunities at regional conferences led by Girls Inc. national Training manager main contact at national

11. Ongoing Support Girls Incorporated partners are key Part of Girls Inc. national�s successful approach of scaling and sustaining STEM programs. Prestige within Girls Inc. to be a curriculum pilot site. Provide guidance to other sites. Develop Program Adaptation Toolkit To be developed and used by Girls Inc. national and affiliates. The Toolkit will include a self-assessment of readiness to implement Build IT program support suggestions (e.g. funding) scenarios based on site contexts (e.g. rural vs. urban) PD guides and contacts for nationally run PD evaluation tools.

12. Research Questions Is the Build IT curriculum adoptable and adaptable in different settings? How is the curriculum adapted to work effectively in different settings? Are girls engaged, achieving IT fluency, and interested in pursuing IT careers, including taking high school mathematics and computer science courses necessary to pursue these careers? Is the Build IT curriculum sustainable in different settings? Is staff capacity at each site increased and supported in order to offer this IT fluency programming?

13. A Framework for Scaling Is the Build IT curriculum adoptable and adaptable in different settings? How is the curriculum adapted to work effectively in different settings? Fidelity Are girls engaged, achieving IT fluency, and interested in pursuing IT careers, including taking high school mathematics and computer science courses necessary to pursue these careers? Spread Is the Build IT curriculum sustainable in different settings? Sustainability Is staff capacity at each site increased and supported in order to offer this IT fluency programming? Program ownership

14. Fidelity �Mutual adaptation� To what depth are the sites implementing the key elements of the Build IT program Implementation framework Unpack these questions using a framework proposed by Cynthia Coburn and included in Volume II Scale up in Education Issues in PracticeUnpack these questions using a framework proposed by Cynthia Coburn and included in Volume II Scale up in Education Issues in Practice

15. Spread The numbers During the pilot, approximately 210 girls reached. Five years after the pilot, 20,000 girls reached each year through the Girls Inc. network. Beliefs, norms and principles To what extent are the enduring understandings, performance tasks, and interactions with IT professionals being used? Is there hands-on and time for reflection on enduring understandings?

16. Sustainability Key elements and instructional approach permeates all levels: National organization (professional development and curriculum support staff) Affiliate (executive director and managers) Program site (program leaders)

17. Shift in Program Ownership From SRI and Girls Inc. of Alameda County to National organization Affiliates Program staff

18. Evaluation Goals: Discover how girls� attitudes and knowledge are changing Understand implementation in different contexts Analyze staff capacity and identify support needs Provide feedback on scaling efforts to Girls Inc. National, GIAC, and SRI as well as Girls Inc. staff at each site. Share understandings from scaling effort with STEM community. Three types of evaluation: Summative Formative Self-evaluation

19. Summative Evaluation Led by HTA Methods: IT Attitudes Survey IT Concepts Survey Interviews with SRI, Girls Inc. staff, and affiliates

20. Formative Evaluation Led by SRI, with local evaluators for each affiliate site (graduate students) Centered on an implementation rubric outlining high-, medium-, and low-quality implementation (based on experiences at GIAC) Methods: Observations of sessions Interviews with girls Interviews with Girls Inc. staff (coordinated with summative evaluation interviews)

21. Self-evaluation Led by Girls Inc. staff at each site Each site determines how much and what kind of self-evaluation will serve their needs SRI-created set of self-evaluation tools: Professional development post-training surveys Observation protocols for girls� learning and staff capacity Coordinator/leader planning check-in form End-of-unit reflection form Implementation rubric

22. Discussion