1 / 70

Patterns in Education: Linking Theory to Practice

Patterns in Education: Linking Theory to Practice. Theodore Frick Department of Instructional Systems Technology School of Education Indiana University Bloomington. Overview of APT&C. Analysis of Patterns in Time and Configuration: APT&C

miracle
Download Presentation

Patterns in Education: Linking Theory to Practice

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Patterns in Education: Linking Theory to Practice Theodore Frick Department of Instructional Systems Technology School of Education Indiana University Bloomington Patterns in Education, AECT 2006

  2. Overview of APT&C • Analysis of Patterns in Time and Configuration: APT&C • Fundamental change in perspective for measurement and analysis • Bridges quantitative and qualitative paradigms • APT for temporal patterns (both joint and sequential occurrences of events) • APC for structural patterns (configurations) Patterns in Education, AECT 2006

  3. Overview cont’d • APT&C based on mathematical theories and general systems theory • Value of APT&C is that results can be directly related to practice • Through APT&C we have new ways of conducting educational research Patterns in Education, AECT 2006

  4. Outline of this presentation • The dilemma: qualitative vs. quantitative methodologies • Three examples of empirical studies that used APT&C: • Academic learning time (APT joint occurrences) • Patterns of mode errors in human-computer interfaces (APT sequential occurrences) • Student autonomy structures in a Montessori classroom (APC patterns of student choice of work and guidance of learning) Patterns in Education, AECT 2006

  5. Quantitative vs. Qualitative Paradigms • Research methods in 20th century were largely quantitative. • Qualitative and mixed methods are gaining more use in research during past two decades. • Main problems: • Quantitative methods seldom yield significant results that can be directly linked to educational practice (due to large within-group variances in experiments or treatments) • Qualitative methods can provide good insights into practice, but conclusions are often restricted (low generalizability due to sampling strategy, and may or may not transfer to similar situations) Patterns in Education, AECT 2006

  6. Three Empirical Studies to Illustrate Value of APT&C • Academic learning time of mildly handicapped children (Frick, 1990) • Patterns of mode errors in human-computer interfaces (An, 2003) • Student autonomy structures in a Montessori classroom (Koh, 2006) Patterns in Education, AECT 2006

  7. Study # 1:Academic Learning Time Study • 25 systems observed in central and southern Indiana • Tracked 25 target students in academic activities over several months for 8 -10 hours each • Trained observers coded types of academic learning contexts, task difficulty and task success • Observers also coded student and instructor behaviors in math and reading (about 500 time samples at one-minute intervals for each target student) • Nearly 15,000 time moments sampled overall. Patterns in Education, AECT 2006

  8. What observers coded in math and reading activities each minute • Types of student engagement: written, oral, and covert on-task; off-task behaviors (later recoded as engagement, EN, and non-engagement, NE) • Types of instructor behaviors: structuring, explaining, demonstrating, questioning, feedback (later recoded as direct instruction, DI), and monitoring academic seatwork (non-direct instruction, ND). • Observer comments to elaborate what was happening Patterns in Education, AECT 2006

  9. Observer coding form Patterns in Education, AECT 2006

  10. Codes for target student moves Patterns in Education, AECT 2006

  11. Codes for instructor moves and focus Patterns in Education, AECT 2006

  12. Standard analysis: columns 1 and 2: independent measures of DI and of EN were correlated (n = 25) Patterns in Education, AECT 2006

  13. Linear Models Approach • Linear models approach (quantitative method): • Relates independent measures through a mathematical function • Treats deviation from model as error variance Patterns in Education, AECT 2006

  14. Linear Models Approach cont’d Patterns in Education, AECT 2006

  15. Linear models results: • Means and standard deviations • Mean p(DI) = 0.432 s.d. = 0.144 • Mean p(EN) = 0.741 s.d. = 0.101 • Regression equation • EN = 0.57 + 0.40DI • R2 = 0.33 • DI “explains” 33 percent of the variance in student engagement; 67 percent unexplained Patterns in Education, AECT 2006

  16. Analysis of Patterns in Time • APT measures a relation directly by counting occurrences of when a temporal pattern is true or false in observational data • Probability of joint or sequential occurrence can be estimated for a pattern from the counts Patterns in Education, AECT 2006

  17. APT Results for same 25 systems: includes measures of joint and conditional occurrences Patterns in Education, AECT 2006

  18. APT Results • Means and standard deviations for the relations • Mean p(EN | DI) = 0.967 s.d. = 0.029 • Mean p(EN | ND) = 0.573 s.d. = 0.142 • When direct instruction is occurring, students are highly engaged. • When non-direct instruction is occurring they are less engaged. • Students were 13 times more likely to be off-task during non-direct instruction compared with direct instruction: (1 - 0.573) / (1 – 0.967) = 12.94. Patterns in Education, AECT 2006

  19. APT: joint occurrence calculation example p(DI) = ¾ = 0.75 p(ND) = ¼ = 0.25 p(EN) = ½ = 0.50 p(NE) = ½ = 0.50 p(DI & EN) = 2/4 = 0.50 p(DI & NE) = ¼ = 0.25 p(ND & EN) = 0/4 = 0.0 p(ND & NE) = ¼ = 0.25 p(EN|DI) = 2/3 = 0.67 p(EN|ND) = 0/1 = 0.00 Patterns in Education, AECT 2006

  20. LMA vs. APT • Linear models relate the independent measures by a function for a line: • e.g., EN = 0.57 + 0.40DI • APT measures the relation in terms of joint, conditional, or sequential occurrence: • e.g., p (EN|DI) = 0.967 • e.g., p (EN|ND) = 0.573DI = direct instruction, EN = student engagement, ND = non-direct instruction Patterns in Education, AECT 2006

  21. Study #2:Patterns of Mode Errors in HCI • Software mode: when the same action results in two or more outcomes (Raskin, 2000). • E.g., In one context, pressing the ‘d’ key results in the letter ‘d’ echoed on the screen • In another context, pressing the ‘d’ key results in deleting a file. • Mode errors by humans can cause serious problems: • Destruction of important work • Decreased productivity • Not able to complete tasks • Modes occur in almost all modern human-computer interfaces (e.g., OS 10, Windows XP, Word, Photoshop, etc.) Patterns in Education, AECT 2006

  22. An (2003) study of mode errors • Mixed methods approach (usability evaluation, qualitative and quantitative) • 16 college students performed eight computer tasks with three modern GUI interfaces (word processor, address book, image editor). • Participants were videotaped, and stimulated- recall interviews were conducted immediately afterwards to clarify why certain actions were taken, when viewing their videos. Patterns in Education, AECT 2006

  23. An (2003) study of mode errors (cont’d) • Over 280 problematic actions were observed, and 52 were problems due to mode errors • 52/280 = .19, or roughly 1 out of 5 problems were due to software modes • Three general patterns (conditions) of mode errors emerged from qualitative analyses: • Type A: Right action, wrong result • Type B: It isn’t there where I need it • Type C: It isn’t there at all Patterns in Education, AECT 2006

  24. An (2003) study of mode errors (cont’d) • Source of error analysis revealed that mode errors appeared to result from 8 types of design incongruity: • Unaffordance • Invisibility • Misled expectation • Unmet expectation • Mismatched expectation • Inconsistency • Unmemorability • Over-automation Patterns in Education, AECT 2006

  25. An (2003) study of mode errors (cont’d) • Consequences of mode errors: • Can’t find hidden function • Can’t find unavailable function • False success • Stuck performance • Inhibited performance • Inefficient performance Patterns in Education, AECT 2006

  26. APT: analysis of sequential patterns of mode errors, sources and consequences Patterns in Education, AECT 2006

  27. APT: analysis of sequential patterns of mode errors, sources and consequences Patterns in Education, AECT 2006

  28. APT: analysis of sequential patterns of mode errors, sources and consequences • APT results have practical implications • E.g., if the mode error is ‘right action, wrong result’ and if the source of the error is unaffordance (function not obvious), then 67 percent of the time users could not find a hidden function or thought they did the task correctly when in fact they had not (false success). Patterns in Education, AECT 2006

  29. APT Methodology: sequential occurrence • When one event precedes another, and when observers code the order in which events occur: • APT can estimate the probability of the consequent following the antecedent event. • APT can estimate likelihoods of sequences longer than two (unlike Markov chains). • APT can estimate both joint and sequential event occurrences in complex combinations. Patterns in Education, AECT 2006

  30. APT Coding (temporal configuration) Patterns in Education, AECT 2006

  31. APT Classifications and Categories • Each column is a classification • Classifications co-exist in time • Categories of events within a classification cannot co-exist in time (since they are mutually exclusive, by definition) • An observer codes event changes within each classification in the order that they occur. • Date/time is always a classification and is recorded whenever there is an event change. Patterns in Education, AECT 2006

  32. Example of sequential coding with three classifications Patterns in Education, AECT 2006

  33. APT Query: IF target student IS Mona? Patterns in Education, AECT 2006

  34. APT Query and Results Query IF target student IS Mona? Results Cumulative duration = (9:13 – 9:01) = 12 minutes Cumulative frequency = 1 event Likelihood = 1 out of 1 relevant event changes = 1.00 Proportion time = 12 minutes out of 12 = 1.00 Patterns in Education, AECT 2006

  35. APT Query: IF target student is Mona AND instruction is direct? Patterns in Education, AECT 2006

  36. APT Query Results Query IF target student IS Mona AND instruction IS direct? Results Cumulative duration = (9:08 – 9:01) = 7 minutes Cumulative frequency = 1 event Likelihood = 1 out of 2 relevant event changes = 0.50 Proportion time = 7 minutes out of 12 = 0.583 Patterns in Education, AECT 2006

  37. APT Query: IF target student IS Mona AND instruction IS direct, THEN student engagement IS on-task? Patterns in Education, AECT 2006

  38. APT Query Results Query IF target student IS Mona AND instruction IS direct, THEN student engagement IS on-task? Results Cumulative duration = (9:06 – 9:03) + (9:08 – 9:07) = 4 minutes Cumulative frequency = 2 Likelihood = 2 out of 4 = 0.50 Proportion time = 4 minutes out of 6 = 0.667 Patterns in Education, AECT 2006

  39. APT Query Syntax Patterns in Education, AECT 2006

  40. APT Syntax (cont’d) Patterns in Education, AECT 2006

  41. APT Syntax (cont’d) Patterns in Education, AECT 2006

  42. APT Query Syntax • Thus, simple to very complex temporal patterns can be specified within APT queries. • Joint and/or sequential occurrences of events can be specified. • Results include frequency counts, likelihood estimates, durations and proportions of total time. Patterns in Education, AECT 2006

  43. Theoretical Foundationsof APT • Mathematical theory • Set theory • Probability theory • Information theory • Classifications (more than one, non-exclusive) • Categories within each classification must be mutually exclusive and exhaustive • General systems theory • SIGGS Theory Model Patterns in Education, AECT 2006

  44. Advantages of APT • APT brings theoretical rigor to pattern identification in qualitative research. • APT measures relations not possible in quantitative methods such as the linear models approach. • APT requires a different kind of conceptual framework for measurement and analysis than those for qualitative and quantitative approaches. Patterns in Education, AECT 2006

  45. APC: Analysis of Patterns in Configuration • Thompson (2005) realized that APT could be extended to measure and analyze structure of systems. • Structure pertains to relationships among parts. Patterns in Education, AECT 2006

  46. Familiar Patterns: Structural • Geographical relation: • Bloomington is located in southern Indiana on the North American continent. • Bloomington is south of Indianapolis. • Organizational relation: • Gerardo Gonzalez is University Dean of the School of Education who directs and supervises: • Peter Kloosterman, Executive Associate Dean, SoE, IUB campus • Khaula Murtahda, Executive Associate Dean, SoE, IUPUI campus Patterns in Education, AECT 2006

  47. Familiar Patterns: Structural • Familial relation: • Philip and Irma Frick are the parents of Theodore Frick • William and Helen Brophy are the parents of Kathleen Brophy • Instructional relation: • During fall semester, 2005,T. Frick was the R690 instructor of: • Andrew, Omer, Shyamasri, Nichole, Jamison, Sunnie, Emmanuel, Uvsh, Chris, Theano Patterns in Education, AECT 2006

  48. A rel B A pattern is a relation • General form of a relation: Patterns in Education, AECT 2006

  49. Temporal & Structural Patterns & Logical Relations • Temporal Patterns • A precedes B • A co-occurs with B • Structural Patterns or Configurations • A affect relation B • Logical Relations • A implies B • A is equivalent to B Patterns in Education, AECT 2006

  50. Affect relation: guides research of Faculty Person 2 Faculty Person 1 Student 3 Student 1 Student 4 Student 5 Student 2 Old IST Ph.D. structure Patterns in Education, AECT 2006

More Related