scientifically based research a guide for educational publishers and developers practical issues l.
Download
Skip this Video
Download Presentation
Scientifically Based Research: A Guide for Educational Publishers and Developers Practical Issues

Loading in 2 Seconds...

play fullscreen
1 / 37

Scientifically Based Research: A Guide for Educational Publishers and Developers Practical Issues - PowerPoint PPT Presentation


  • 116 Views
  • Uploaded on

Scientifically Based Research: A Guide for Educational Publishers and Developers Practical Issues. Overview. Cost considerations How long will it take? Securing school participation in research projects Developing research white papers and portfolios. Cost Considerations. Rule of thumb:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Scientifically Based Research: A Guide for Educational Publishers and Developers Practical Issues' - leyna


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
scientifically based research a guide for educational publishers and developers practical issues
Scientifically Based Research:A Guide for Educational Publishers and DevelopersPractical Issues

Interactive Educational Systems Design, Inc., 2003

overview
Overview
  • Cost considerations
  • How long will it take?
  • Securing school participation in research projects
  • Developing research white papers and portfolios

Interactive Educational Systems Design, Inc., 2003

cost considerations
Cost Considerations

Rule of thumb:

  • About 10% of a development project’s budget should be set aside for studying impact.

Interactive Educational Systems Design, Inc., 2003

cost considerations4
Cost Considerations
  • There are no general rules of thumb about the relative cost of different research designs.

Interactive Educational Systems Design, Inc., 2003

cost considerations5
Cost Considerations

Typical labor costs:

  • Senior researcher time
  • Project manager time
  • Site coordinator time
  • Data collector time
  • Data analyst time

Interactive Educational Systems Design, Inc., 2003

cost considerations6
Cost Considerations

Costly labor intensive tasks:

  • Direct observations
  • Scoring of performances
  • Sometimes, data analyses

Interactive Educational Systems Design, Inc., 2003

cost considerations7
Cost Considerations

Observation costs:

  • Traveling to observation sites
  • Identifying or developing observation protocols
  • Developing training materials
  • Training observers
  • Re-checking observer reliability

Interactive Educational Systems Design, Inc., 2003

cost considerations8
Cost Considerations

Ways to cut observation costs:

  • Use local observers
  • Use graduate student observers
  • Use publicly available observation protocols and training materials

Interactive Educational Systems Design, Inc., 2003

cost considerations9
Cost Considerations

Travel costs related to:

  • Meetings between researchers and the publisher or developer
  • Meetings between senior researcher(s) and leaders of participating institutions
  • On-site data collection

Interactive Educational Systems Design, Inc., 2003

cost considerations10
Cost Considerations

Performance scoring costs:

  • Development of scoring rubrics and performance prompts
  • Training scorers
  • Conducting reliability checks

Interactive Educational Systems Design, Inc., 2003

how long will it take
How Long Will It Take?
  • Conduct literature review 2-3 months
  • Design research plan 1 month
  • Design and test new instruments (if any) 2-3+ months
  • Pilot test research design, analyze data, and revise (time permitting) 6-8 months
  • Implement study 4-10 months
  • Data analysis 1-2 months
  • Report preparation 1-2 months

Interactive Educational Systems Design, Inc., 2003

how long will it take12
How Long Will It Take?

Saving time on the literature review:

  • Search for a meta-analysis of experimental studies on a topic.
  • Search for an existing literature review.
  • Locate overview publications published by nationally recognized professional organizations.
  • Conduct electronic searches.
  • Consult a research librarian.

Interactive Educational Systems Design, Inc., 2003

how long will it take13
How Long Will It Take?

Study duration depends on:

  • The nature of the product
  • The scope of the product or of the “chunks” of the product to be evaluated
  • How much new data collection is necessary
  • The nature of the data analysis

Interactive Educational Systems Design, Inc., 2003

securing school participation in research projects
Securing School Participation in Research Projects
  • Focus on the key district decision-maker

Interactive Educational Systems Design, Inc., 2003

securing school participation in research projects15
Securing School Participation in Research Projects

Before consenting to participation, most

district decision-makers will want to know:

  • The benefits to participation
  • The costs
  • That no harm will come to participants

Interactive Educational Systems Design, Inc., 2003

securing school participation in research projects16
Securing School Participation in Research Projects

It may be necessary to:

  • Obtain informed consent from students, parents, and/or teachers
  • Receive approval from the District’s Internal Review Board for the Protection of Human Subjects

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios
Developing Research White Papers and Portfolios

Why Publishers Need a White Paper or Portfolio

  • District administrators need to document that their instructional materials purchase decisions are based on research.
  • Publishers need to make the best case that their product designs are supported by SBR.

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios18
Developing Research White Papers and Portfolios

Research White Paper

Hybrid research and marketing document

  • Executive summary of a product’s research base
  • Description of related product features
  • Direct research on the product’s effectiveness (if available)

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios19
Developing Research White Papers and Portfolios

Research Portfolio

Detailed abstracts of the research cited in

the white paper, including:

  • Supporting research
  • Direct research of the product’s effectiveness

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios20
Developing Research White Papers and Portfolios

Researching the Product Design

Two alternative scenarios:

  • Products currently being designed
  • Existing products

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios21
Developing Research White Papers and Portfolios

Researching the Product Design

For products currently being designed:

  • Build a review of the research literature into the design process.
  • Update the review just prior to preparing the white paper or portfolio.

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios22
Developing Research White Papers and Portfolios

Researching the Product Design

For existing products:

  • Draw on the history of the product design.
  • If there is no historical record, conduct a “reverse analysis” of the product design.

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios23
Developing Research White Papers and Portfolios

Researching the Product Design

When completing a literature review,

distinguish between:

  • Theories or expert opinion
  • Scientifically based research

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios24
Developing Research White Papers and Portfolios

What to Cover in the White Paper

  • Direct SBR evidence of product effectiveness
  • Content coverage

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios25
Developing Research White Papers and Portfolios

What to Cover in the White Paper

  • Instructional design
  • Reference list

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios26
Developing Research White Papers and Portfolios

Expanding to Include a Research Portfolio

  • More detailed abstracts of each study featured in the white paper
  • Can be distributed in print and online
  • Will help schools and districts make the case that their purchase decisions include consideration of SBR

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios27
Developing Research White Papers and Portfolios

Expanding to Include a Research Portfolio

Each abstract should include:

  • Description of the student population
  • Number of students, classes, and schools
  • Explanation of how the sample was selected

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios28
Developing Research White Papers and Portfolios

Expanding to Include a Research Portfolio

Each abstract should include:

  • Description of the experimental treatment
  • Description of the comparison treatment
  • Description of the assessment instrument(s)

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios29
Developing Research White Papers and Portfolios

Expanding to Include a Research Portfolio

Each abstract should include:

  • Statistical analyses performed
  • Summary of results of analyses
  • Author’s conclusions
  • Limitations of the study
  • Links to the target product

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios30
Developing Research White Papers and Portfolios

Size of Documents

  • Research White Paper:
    • Supplemental product: 5-10 pages
    • Core curriculum product: 10-20 pages
  • Research Portfolio: 1-2 pages per abstract

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios31
Developing Research White Papers and Portfolios

Who Should Help Prepare the White Paper?

  • In-house research expert or outside research consultancy
  • Product design staff
  • Marketing/communication staff

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios32
Developing Research White Papers and Portfolios

Cost Factors

Cost of three major phases:

  • Research review
  • White paper development
  • Portfolio abstract development

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios33
Developing Research White Papers and Portfolios

Cost Factors

Research review tasks:

  • Search literature
  • Develop criteria for including research in the white paper
  • Complete preliminary review of research documents
  • Write internal abstracts (not for publication)

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios34
Developing Research White Papers and Portfolios

Cost Factors

Research review tasks:

  • Review product information (if the writer is an external contractor)
  • Prepare outline of white paper
  • Make final decisions about white paper development

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios35
Developing Research White Papers and Portfolios

Cost Factors

White paper development tasks:

  • Write summaries of individual research studies
  • Write explanations of how product content, features, and functions relate to the research
  • Write the introduction and conclusion
  • Prepare the reference list
  • Edit the paper

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios36
Developing Research White Papers and Portfolios

Cost Factors

Portfolio of abstracts development tasks:

  • Develop template of information categories (data fields)
  • Write abstracts for publication
  • Edit the abstracts

Interactive Educational Systems Design, Inc., 2003

developing research white papers and portfolios37
Developing Research White Papers and Portfolios

How Long Will It Take?

  • Research review 1-3 months
  • White paper development 2-4 weeks
  • Portfolio of abstracts development 1-3 weeks

Interactive Educational Systems Design, Inc., 2003