1 / 28

Improving and Assessing Instructional Effectiveness

Improving and Assessing Instructional Effectiveness. Theodore Frick Department of Instructional Systems Technology School of Education Indiana University Bloomington Invited Lecture to IUB Medical Sciences Faculty October 14, 2002. Overview.

Download Presentation

Improving and Assessing Instructional Effectiveness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Improving and Assessing Instructional Effectiveness Theodore Frick Department of Instructional Systems Technology School of Education Indiana University Bloomington Invited Lecture to IUB Medical Sciences Faculty October 14, 2002

  2. Overview • Confounding of instructional effectiveness by other factors • Types of knowledge about education • Type 2, 3 and 4 examples • Recommendations Improving and Assessing Instructional Effectiveness -- Ted Frick

  3. The Logic of Effectiveness Improving and Assessing Instructional Effectiveness -- Ted Frick

  4. Confounding Factors • Learning occurs without intentional instruction. • Learning occurs despite poor instruction. • We’re not the only teachers a student has. • Results of instruction may occur LONG afterwards. Improving and Assessing Instructional Effectiveness -- Ted Frick

  5. Conflation of Research Methods and Outcomes • In addition to the confounding problem in determining effectiveness of instruction, there has been a lot of debate in education about inquiry methods – e.g., quantitative vs. qualitative. • Continuing problem of bridging research and practice in education Improving and Assessing Instructional Effectiveness -- Ted Frick

  6. Consider this Debate Imagine for a moment three carpenters arguing about which tool is best. • Quanta: "Hammers and nails are clearly superior." • Qualia: "I disagree.  Screwdrivers and screws are much more effective." • Performa: "You're both wrong.  Saws are best for cutting wood." • Quanta: "Who said anything about cutting wood?  I thought we were talking about fastening wood together." • Qualia: "Right. Who needs saws?" • Performa: "I do. I need to cut this board in half." Improving and Assessing Instructional Effectiveness -- Ted Frick

  7. Premise The kind of knowledge about education that we create through disciplined inquiry determines what research methods are appropriate and useful. Improving and Assessing Instructional Effectiveness -- Ted Frick

  8. Types of Educology: Knowledge about Education • Methodology of Theory Building (Elizabeth Steiner, 1988) (link) • Non-axiological knowledge – what is • Axiological knowledge – pertaining to values: • Instrumental value: good for – what is effective • Intrinsic value: good in itself – what is worthwhile • The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles (Lee Cronbach, et al., 1972) Improving and Assessing Instructional Effectiveness -- Ted Frick

  9. Six Types of Knowledge (Frick) See Frick: R690 Syllabus for further details Improving and Assessing Instructional Effectiveness -- Ted Frick

  10. Six Types of Knowledge (Frick) Example: Type 1 Knowledge Claim ‘Elizabeth Steiner is an educational philosopher.’ Improving and Assessing Instructional Effectiveness -- Ted Frick

  11. Six Types of Knowledge (Frick) Example:Type 2 Knowledge Claim ‘Students who spend more time engaged successfully in arithmetic and reading tasks score more highly on achievement tests in those content areas.’ (Fisher, et al., 1976) Improving and Assessing Instructional Effectiveness -- Ted Frick

  12. Six Types of Knowledge (Frick) Example:Type 3 Knowledge Claim ‘The DISTAR program for teaching arithmetic and reading to elementary students works.’ Improving and Assessing Instructional Effectiveness -- Ted Frick

  13. Six Types of Knowledge (Frick) Example:Type 4 Knowledge Claim ‘Instruction is most effective when it is problem-based, activates prior learner knowledge, demonstrates what is to be learned, provides opportunities for guided practice, and encourages integration with everyday life.’ (Merrill, 2001) Improving and Assessing Instructional Effectiveness -- Ted Frick

  14. Six Types of Knowledge (Frick) Example:Type 5 Knowledge Claim ‘The corporal punishment policy in Houston schools is a bad policy.’ Improving and Assessing Instructional Effectiveness -- Ted Frick

  15. Six Types of Knowledge (Frick) Example:Type 6 Knowledge Claim ‘Teachers and students should respect each other.’ Improving and Assessing Instructional Effectiveness -- Ted Frick

  16. Do not confuse ends and means The kind of knowledge about education that we create through disciplined inquiry determines what research methods are appropriate and useful. • Outcome of research is knowledge. • Inquiry methods are a means to that end. • Criteria for evaluating adequacy of research methodology are NOT the same for each knowledge type – e.g., • criteria for statistical inference to a broad population do not apply to Types 1, 3, and 5; • effectiveness not of concern in Types 1, 2, 5, and 6, but main concern in 3 and 4. Improving and Assessing Instructional Effectiveness -- Ted Frick

  17. For today… Focus on a few examples of Types 2, 3 and 4 knowledge of education • Type 2: Academic learning time research • Type 3 outcomes: Program/product evaluation – Kirkpatrick’s 4 levels (1959) • Type 4 process: Effective Web Instruction – Frick & Boling (2002) • Type 4: Principles of 5-Star Instruction – David Merrill (2001) Improving and Assessing Instructional Effectiveness -- Ted Frick

  18. Six Types of Knowledge (Frick) Type 2 Example: Relationship of academic learning time and student achievement Improving and Assessing Instructional Effectiveness -- Ted Frick

  19. Type 2: Academic Learning Time ALT is positively correlated with academic achievement in the same content areas (Fisher, et al., 1976; Rieth & Frick, 1983; Berliner; 1985). Improving and Assessing Instructional Effectiveness -- Ted Frick

  20. Six Types of Knowledge (Frick) Type 3: Evaluation of a particular instructional program or product Improving and Assessing Instructional Effectiveness -- Ted Frick

  21. Type 3: Program/Product Evaluation – Kirkpatrick’s Levels Techniques for Evaluating Training Programs -- Donald Kirkpatrick (1959) • Reaction (satisfaction) • Learning (achievement) • Behavior (transfer to real context) • Results (impact on organization/context) More on Kirkpatrick’s levels from Encyclopedia of Educational Technology Improving and Assessing Instructional Effectiveness -- Ted Frick

  22. Six Types of Knowledge (Frick) Type 4 Example: how to create effective instructional products – design theory Improving and Assessing Instructional Effectiveness -- Ted Frick

  23. Type 4: Effective Web Instruction: An Inquiry-Based Process – Frick & Boling More Improving and Assessing Instructional Effectiveness -- Ted Frick

  24. Type 4: First Principles of Instruction – David Merrill Integration Activation Problem Application Demonstration Improving and Assessing Instructional Effectiveness -- Ted Frick

  25. Type 4: First Principles of Instruction (cont’d) “Learning is facilitated when: • Learners are engaged in solving real-world problems. • Existing knowledge is activated as a foundation for new knowledge. • New knowledge is demonstrated to the learner. • New knowledge is applied by the learner. • New knowledge is integrated into the learner’s world.” (Merrill, 2001, p. 2) Does your instruction rate 5 stars? A rating scale Improving and Assessing Instructional Effectiveness -- Ted Frick

  26. Six Types of Knowledge (Frick) Type 3: Indiana University physician education program Improving and Assessing Instructional Effectiveness -- Ted Frick

  27. Recommendations In physician education, focus on Type 3 knowledge: • Apply Merrill’s 5 principles as criteria for design. See video (requires RealPlayer). • Use Kirkpatrick’s 4 levels of evaluation for your programs at IU. Improving and Assessing Instructional Effectiveness -- Ted Frick

  28. Questions? • This presentation is available at: http://www.indiana.edu/~tedfrick/improveinstruction.ppt • Contact: frick@indiana.edu Improving and Assessing Instructional Effectiveness -- Ted Frick

More Related