Predicting Outcomes of Systemic Change in Education

1. Predicting Outcomes of Systemic Change in Education Theodore Frick Kenneth R. Thompson Joyce Koh Predicting Outcomes of Systemic Change in Education

2. The Problem: No Adequate Theory • K-12 schools under increasing pressure with NCLB (No Child Left Behind) • Failing schools will have real incentive to change • Questions are: Change what? Change how? • Attempts to change will be trial and error if no valid theory • What we need is good educational systems theory Predicting Outcomes of Systemic Change in Education

3. Bridge Analogy • Consider an old bridge that is failing • Structurally weak • Too few lanes for traffic • Traffic increasing • If not fixed, will collapse: vehicles plunge into river • Would we build a new bridge by trial and error? Predicting Outcomes of Systemic Change in Education

4. Bridge Analogy Cont’d • No, we would not build a bridge by trial and error! • Modern engineers use proven scientific theories: Newtonian physics, classical mechanics, statics, structural engineering • But in education, we are attempting to change systems by trial and error Predicting Outcomes of Systemic Change in Education

5. We Need Educational Systems Theory • Lewin: “There is nothing so practical as good theory.” • We have theories of: • Instruction • Learning • Pedagogy, e.g., Montessori method • We have had no educational systems theory Predicting Outcomes of Systemic Change in Education

6. ATIS: Axiomatic Theories of Intentional Systems • Until now! • ATIS provides a robust, complex theory that can be applied to educational systems • See Thompson (2006) seminal articles in Scientific Inquiry Journal: http://www.iigss.net/Scientific-Inquiry/table.htm • Foundations and definitions • Methodology of theory construction Predicting Outcomes of Systemic Change in Education

7. NCLB Example • To make this more concrete, consider the following scenario: Smithtown School #9 failed to achieve state standards for No Child Left Behind (NCLB) Predicting Outcomes of Systemic Change in Education

8. SMITHTOWN SCHOOL #9 Scenario Parents start transferring children to other schools Predicting Outcomes of Systemic Change in Education

9. Then filtration increases If input decreases SMITHTOWN SCHOOL #9 This is a FAILING school. Tommy shouldn’t enroll here! Enrollment falls ATIS Prediction – Axiom 13 NCLB rating deters enrollment Predicting Outcomes of Systemic Change in Education

10. Then storeput decreases If input decreases Enrollment falls ATIS Prediction – Axiom 11 Fewer students attending classes Predicting Outcomes of Systemic Change in Education

11. Then fromput decreases If input decreases Hmm…there aren’t as many diplomas to print this year! ADMINISTRATION OFFICE Enrollment falls ATIS Prediction – Axiom 10 Fewer students to graduate Predicting Outcomes of Systemic Change in Education

12. Then feedout decreases If input decreases Enrollment falls ATIS Prediction – Axiom 16 Fewer graduates Predicting Outcomes of Systemic Change in Education

13. If filtration increases Then adaptability increases SMITHTOWN SCHOOL #9 BOARD MEETING AGENDA: How to improve achievement scores? This is a FAILING school. Tommy shouldn’t enroll here! NCLB rating deters enrollment ATIS Prediction – Axiom 28 SMITHTOWN SCHOOL #9 Smithtown adapts to maintain system stability Predicting Outcomes of Systemic Change in Education

14. Using ATIS with Smithtown’s adaptation strategies How can Smithtown adapt? Change the structure (cf. Senge, 2006; Thompson, 2006) What if Smithtown changes its structure by increasing STRONGNESS of affect relations that are of type: guidance of learning? Predicting Outcomes of Systemic Change in Education

15. Smithtown’s proposed strategy • Structural change: Increase avenues of instruction through: • Teaching aides • Peer tutoring • Instructional technology e.g. using e-Learning software Increase strongness Predicting Outcomes of Systemic Change in Education

16. If strongness increases Then hierarchical order decreases Teaching aides Teachers GUIDE GUIDE GUIDE GUIDE GUIDE E-learning software Peer tutoring GUIDE ATIS Prediction – Axiom 56 More ‘guidance of learning’ connections for students Before: Teacher is main guide. After: Less focus on teacher as guide of learning. Predicting Outcomes of Systemic Change in Education

17. Then flexibility increases If strongness increases Teaching aides Teachers Teaching aides Teachers E-learning software Peer tutoring Peer tutoring E-learning software More ‘guidance of learning’ connections for students ATIS Prediction – Axiom 55 More different ways for guiding learning of students Predicting Outcomes of Systemic Change in Education

18. Then filtration decreases If strongness increases Teaching aides Teachers SMITHTOWN SCHOOL #9 FAILURE E-learning software Peer tutoring More ‘guidance of learning’ connections for students SUCCESS ATIS Prediction – Axiom 108 They’ve made AYP. Tommy can enroll here! Smithtown #9 makes NCLB rating. This encourages enrollment. AYP = Annual Yearly Progress (part of NCLB law) Predicting Outcomes of Systemic Change in Education

19. Then isomorphism increases If strongness increases SMITHTOWN SCHOOL #9 They’ve improved achievement scores and made AYP. Tommy can enroll here! SMITHTOWN SCHOOL #5 SMITHTOWN SCHOOL #9 SMITHTOWN SCHOOL #1 SMITHTOWN SCHOOL #12 SMITHTOWN SCHOOL #25 Smithtown #9 makes NCLB rating.This raises enrollment. ATIS Prediction – Axiom 144 Increase strongness Increase strongness SUCCESS Increase strongness Increase strongness Smithtown replicates success strategy for more schools AYP = Annual Yearly Progress (part of NCLB law) Predicting Outcomes of Systemic Change in Education

20. Summary • If we have a valid educational systems theory, • Based on predictable temporal patterns and configurations, • Then we can change an education system with a reasonable expectation that it will actually be improved. Predicting Outcomes of Systemic Change in Education

21. ATIS: Axiomatic Theories of Intentional Systems • Basic strategy (Thompson, 2005) • Affect relations determine system properties. • System properties determine system state. • System state determines the system-descriptive axiom set. • The axiom set provides logic-based predictive outcomes. • Affect relations also determine system topological structure in that every affect relation defines a topology. • Topological structure provides dynamic, real-time predictive outcomes. • System-descriptive axiom set and system topological structure together determine total system structure and system-predictive outcomes. • The logical analysis as dependent upon the system axioms comes first, followed by a topological analysis that establishes in fact the vectored system outcome; that is, that the system is actually taking the path indicated by the logical analysis. Predicting Outcomes of Systemic Change in Education

22. Axiomatic Theories of Intentional Systems – Thompson (2006) • Examples of axioms and theorems: • If system input decreases, then filtration increases. • If system filtration increases, then adaptability increases. • If system strongness increases, then hierarchical order decreases. • If system strongness increases, then flexibility increases. • If system strongness increases, then input increases. • If system strongness increases, then filtration decreases.See full theory (over 200 axioms/theorems) and reports at: http://www.indiana.edu/~aptfrick/reports/ Predicting Outcomes of Systemic Change in Education

23. Applying ATIS to a specific system • Only some of the axioms/theorems in ATIS will apply to a given system. • PESO (Predicting Education System Outcomes) is the software tool that makes predictions, based on specific conditions in a particular system. • In other words, PESO uses ATIS as an expert system. Predicting Outcomes of Systemic Change in Education

24. PESO Demo • Link to PESO Prototype (restricted access) • Enter system condition(s) • PESO applies ATIS and makes predictions. • PESO developed in such a way that additional axioms and theorems can be easily added (Flash ActionScript objects) Predicting Outcomes of Systemic Change in Education

25. APC Demo • Analysis of Patterns in Configuration (Frick & Thompson, 2006) • Specify components and affect relations • APC software calculates values of structural properties • Link to prototype • Uses measures defined in ATIS Graph Theory: http://www.indiana.edu/~aptfrick/reports/11ATISgraphtheory.pdf Predicting Outcomes of Systemic Change in Education

26. Verifying Systems Theory • The systems theory consists of axioms and theorems for making predictions • Axioms and theorems consist of dynamic and structural properties • APT&C can be used as a verification methodology Predicting Outcomes of Systemic Change in Education

27. For ATIS Theory Validation • We need ways of measuring system: • Dynamics (temporal change) • Structures (affect relations) • This will be done by: • Analysis of Patterns in Time • Analysis of Patterns in Configuration • In short: APT&C, see Frick (2005) grant proposal: http://education.indiana.edu/~frick/proposals/apt&c.pdf • This will be covered in the APT&C presentation on Friday at AECT in Dallas. Predicting Outcomes of Systemic Change in Education

28. Using Theoretical Predictions • We can use theoretical predictions to make practical decisions, e.g., • Not smoke, to reduce chances of lung cancer later in life. • Take umbrella if rain is predicted to be highly likely. Predicting Outcomes of Systemic Change in Education

29. Imagine for the moment that… • We have a valid educational systems theory that: • Can predict education systems outcomes based on current conditions (ATIS), and • Is based on empirically verified temporal patterns and configurations in systems Predicting Outcomes of Systemic Change in Education

30. Systemic Change Model • This leads to an inquiry-based systems change strategy: Get Ready >> Set >> Go! Predicting Outcomes of Systemic Change in Education

31. Change Strategy: Get Ready >> Set >> Go! Phase 1:  Get Ready • Identify the specific current education system to be improved. • Over some interval of time, measure system properties (e.g., input, regulation, complexity, strongness) with Analysis of Patterns in Time and Configuration (APT&C) • Use Predicting Educational Systems Outcomes (PESO) software to predict outcomes based on observed system properties under existing conditions (e.g., complexity increases, decreases, or remains constant). These predictions are based on how the system is currently designed and operates under existing conditions, before any new design is implemented. • If these outcomes are what are wanted, then do not modify the system.  Otherwise, proceed to Phase 2. Predicting Outcomes of Systemic Change in Education

32. Change Strategy: Get Ready >> Set >> Go! Phase 2:  Set • Use PESO software to model newly envisioned educational system designs – i.e., the changes desired which are feasible. • Run PESO predictions far out enough in time to make sure all the consequences of the newly designed system would be acceptable.  Are these the wanted outcomes? If yes, proceed to Phase 3. Predicting Outcomes of Systemic Change in Education

33. Change Strategy: Get Ready >> Set >> Go! Phase 3:  Go! • Implement the new design chosen in Phase 2. • Over some interval of time, measure system properties with APT&C. • Verify that predicted system outcomes have occurred.  If not, was something important overlooked in the observation and analysis of this particular system? Proceed to Phase 2. Predicting Outcomes of Systemic Change in Education

34. SimEd Technologies We refer to: • ATIS theory model • APT&C software • PESO software, and the • ‘Get Ready, Set, Go!’ model as SimEd Technologies Predicting Outcomes of Systemic Change in Education

35. SimEd Technologies • Currently under development • Further ATIS theory development • APT and APT software development • PESO development • Need to obtain major funding to support these activities Predicting Outcomes of Systemic Change in Education

36. SimEd Technologies Questions? For more information on SimEd Technologies: http://simedtech.com Predicting Outcomes of Systemic Change in Education