practice guided by research in providing effective student support services l.
Skip this Video
Loading SlideShow in 5 Seconds..
Practice Guided by Research in Providing Effective Student Support Services PowerPoint Presentation
Download Presentation
Practice Guided by Research in Providing Effective Student Support Services

Loading in 2 Seconds...

play fullscreen
1 / 63

Practice Guided by Research in Providing Effective Student Support Services - PowerPoint PPT Presentation

  • Uploaded on

Practice Guided by Research in Providing Effective Student Support Services Prof. Terry Anderson Canada Research Chair in Distance Education Presentation Overview Traditional Opening Joke Need for distance education research Sorry state of current research Methodologies Quantitative

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Practice Guided by Research in Providing Effective Student Support Services' - Rita

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
practice guided by research in providing effective student support services

Practice Guided by Research in Providing Effective Student Support Services

Prof. Terry Anderson

Canada Research Chair in Distance Education

presentation overview
Presentation Overview
  • Traditional Opening Joke
  • Need for distance education research
  • Sorry state of current research
  • Methodologies
    • Quantitative
    • Qualitative
    • Design-based
  • Dissemination and building a research culture
why is distance education better than sex
Why is Distance Education Better than Sex?
  • If you get tired, you can stop, save your place and pick up where you left off.
  • You can finish early without feeling guilty.
  • You can get rid of any viruses you catch with a $50 program from McAfee
  • With a little coffee you can do it all night.
  • You don’t usually get divorced if your spouse interrupts you in the middle of it.
  • And If you're not sure what you are doing, you can always ask your tutor.
defining learner support
“Learner support services” is the variety of non-academic interactions that the student has with a college or university, including:

pre-enrollment services (recruiting, promotion, orientation),

admissions and registration,

academic advising,

program planning,

degree and transcript audit,

technical assistance,

library and bookstore services,

personal and career counseling,

social support services, and

financial planning and management.

(Dirr, 1999)

Open University: “all those elements capable of responding to a known learner or group of learners, before, during and after the learning process” (Thorpe, 2001)

Includes teaching function

Defining Learner Support
defining research
Defining Research
  • “the systematic study of materials and sources in order to establish facts and reach new conclusions” (Oxford Compact Dictionary, 1991).
  • adjectives describing research as
    • disciplined, organized,
    • transparent, problem orientated,
    • public, creative,
    • scientific, systematic,
    • diligent, labourous
    • and accessible
qualities of research
Qualities of Research
  • Characterized by:
    • clear goals;
    • adequate preparation;
    • appropriate methods;
    • significant results;
    • effective presentation and
    • reflective critique.

Glassick, Huber & Maeroff. (1997).

Are such descriptions exclusionary?

why do research on learner services
Why Do Research on Learner Services?
  • Many unresolved questions of traditional distance education
    • - attrition; F2F tutorial value, paced vs unpaced;
  • new forms of distance education provision.
    • what combinations of group based learning are worth the cost and inconvenience?
    • Do face to face tutorials really make a difference or is real time video or audio conferencing just as effective?
    • How much does expensive multimedia really enhance student learning?
    • how important are real time interactions compared to asynchronous ones ?
what makes educational research effective
What Makes Educational Research Effective?
  • Can you think of one research result that you have learned about during the past two days at this conference that will make a difference to your practice?
  • Why does this particular research result make a difference?
    • Question ; context; methodology; clarity of presentation?
why educational research just don t get no respect
Why Educational Research‘Just Don’t Get No Respect’
  • Much research is not valued by either funders, other academics or worse, practitioners
  • Not funded financially
    • Education 0.01 % of expenditures
    • Health 3.0 %
    • High tech companies 10.0 - 15.0 %
    • Overall (Canada, 2002) 1.88%
put in perspective
Put in Perspective
  • International pharmacy giant Pfizer Inc. spent over $200 million (1999) in research related to treatments for animals –
  • 7 times as much as the US government spends on educational research
assessment of de research
Assessment of DE Research
  • Many experimental research projects do not display rigour in their design
  • Many generalize inappropriately
  • Cultural, linguistic and environmental factors often not taken into consideration
  • Few concerned with teacher and tutor support
  • Few studies based on current learning, pedagogical or psychological theories
    • Olugbemiro Jegede (1999)
barriers to educational research
Barriers to Educational Research
  • It’s nobody’s job
    • How many in this room have at least 50% of their job requirement to do research?
  • Negligible industry support
  • No large scale focus on particular problems
  • Nobody keeping score in meaningful terms
  • Pervasive lack of trust in research efficacy
  • In sum, lack of an effective research culture
        • (Burkhardt and Schoenfeld, 2003).
research paradigms
Research Paradigms
  • Quantitative – discovery of the laws that govern behaviour
  • Qualitative – understandings from the inside
  • Design-based – interventions, interactions and their effect
quantitative research
Quantitative Research
  • From Keller to evidence based practice
  • Developed from health research notably Cochrane
  • employs a scientific discourse derived from the epistemologies of positivism and realism.
“those who are seeking the strict way of truth should not trouble themselves about any object concerning which they cannot have a certainty equal to arithmetic or geometrical demonstration”
    • (Rene Descartes,).
  • Inordinate support and faith in randomized controlled studies
quantitative 1 cmc content analysis
Quantitative 1 – CMC Content Analysis
  • Anderson, Garrison, Rourke 1997-2003
    • - 9 papers reviewing results focusing on reliable , quantitative analysis
    • Identified ways to measure teaching, social and cognitive ‘presence’
    • Most reliable methods are beyond current time constraints of busy teachers
    • Maybe basic research as grounding for AI methods of the future
is meta analysis the gold standard
Is meta analysis the gold standard?
  • Two recent Canadian examples
quantitative 2
Quantitative 2
  • Bernard, (2003) How Does Distance Education Compare to Classroom Instruction? A Meta-Analysis of the Empirical Literature Concordia University
  • Results of 2,262 studies examined from 1985-2003, only 232 met comparative quality criteria
quantitative 3 meta analysis
Quantitative 3 – Meta-Analysis
  • Ungerleider and Burns (2003)
  • Systematic review of effectiveness and efficiency of ICT
  • The type of interventions studied were extraordinary diverse –only criteria was a comparison group
  • “Only 10 of the 25 studies included in the in-depth review were not seriously flawed, a sobering statistic given the constraints that went into selecting them for the review.”

Ungerleider, C., & Burns, T. (2003) . A systematic review of the effectiveness and efficiency of networked ICT in education . P.38 Ottawa: Industry Canada. Retireved Jan. 24, 2004 from

quantitative summary
Quantitative Summary
  • Can be useful especially when testing well established practice
  • The need to “control” context often makes results of little value to practicing professionals
  • In times of rapid change too early quantitative testing may mask beneficial positive capacity
  • Will we ever be able to afford blind reviewed, random assignment studies?
qualitative paradigm
Qualitative Paradigm
  • Many different varieties
  • Generally answer the question ‘why’ rather then ‘what’ or ‘how much’?
  • Presents special challenges in DE due to distance between participants and researchers
  • Currently most common type of DE research (Rourke and Szabo, 2002)
1 st qualitative example
1st Qualitative Example
  • Dearnley (2003) Student support in open learning: Sustaining the Process
  • Practicing Nurses, weekly F2F tutorial sessions
  • Phenomenological study using grounded theory discourse
“Support structures to facilitate personal and professional development within this context need to be in place and attention must be given to the provision of effective learner support.” (Dearnley, 2003)
2 nd qualitative example
2nd Qualitative Example
  • Cain; Marrara; Pitre (2003) Support services that matter.
  • Interviews with 8 grad students
  • most students were not likely to take advantage of student support services.
  • students perceived their peers as important sources of academic and social support.
  • students expected their instructor to be a support resource and to be knowledgeable about the on-campus academic and administrative services.
qualitative summary
Qualitative Summary
  • Measure of quality is “critical appraisal concerning plausibility, internal consistency and fit to prevailing wisdom
    • Burkhardt & Schoenfeld (2003
  • But what if the only producers and consumers are researchers not practitioners?
  • Often the only answer that makes it to the trenches is “it depends”
do either qualitative or quantitative methods meet real needs of practicing distance educators
Do Either Qualitative or Quantitative Methods Meet Real Needs of Practicing Distance Educators?
quantitative vs qualitative paradigm wars rekindled
Quantitative vs. QualitativeParadigm Wars Rekindled
  • Current research “more resembles the pendulum swings characteristic of art or fashion, rather than the progressive improvements characteristic of science and technology” (p. 16).
    • Slavin (2002) in Educational Researcher
  • Solution to embrace “evidence based learning”
  • Projected to increase from 5% to 75% of US Gov. funding by 2007 for “research that addresses causal questions and uses random assignments …. As a result funding is likely to substantially increase” Slavin, 2002 p. 15
USA Department of Education (2003) guidelines for Identifying and Implementing Educational Practices Supported By Rigorous Evidence:
push for scientifically based educational research
Push for “Scientifically Based” Educational Research
  • Scientifically based research:
    • employees systematic empirical methods
    • involves rigorous data analysis
    • Relies on measurements or observable methods that provide reliable and valid data across evaluators and observers
    • Is evaluated using experimental or quasi-experimental designs (Eisenhart and Towne 2003)
push for scientifically based educational research36
Push for “Scientifically Based” Educational Research
  • "...means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs"
      • (No Child Left Behind (2001)
quantitative vs qualitative paradigm wars rekindled37
Quantitative vs. QualitativeParadigm Wars Rekindled
  • Dualism marginalizes; stereotypes, separates and backgrounds competing positions
  • We have “been seduced into accepting the inherent value and power differentials that operate in the dualist construction of the 'quantitative/qualitative' binary pair of terms” (Lines, 2001)
is evidence based practice better than opinion based practice
Is evidence based practice better than opinion based practice?
  • No compelling evidence of such.
  • Implementation issues are much more convoluted and socially determined than the mere presentation of evidence.
  • Need to broaden the evidence base and allow results from diverse methodologies to enter the accumulated knowledge base.
but what type of research has most effect on practice
But what type of research has most effect on practice?
  • Kennedy (1999) - teachers rate relevance and value of results from each of major paradigms.
  • No consistent results – teachers are not a homogeneous group of consumers but they do find research of value
  • “The studies that teachers found to be most persuasive, most relevant, and most influential to their thinking were all studies that addressed the relationship between teaching and learning.”
but what type of research has most effect on practice40
But what type of research has most effect on Practice?
  • “The findings from this study cast doubt on virtually every argument for the superiority of any particular research genre, whether the criterion for superiority is persuasiveness, relevance, or ability to influence practitioners’ thinking.” Kennedy, (1999)
“In any dispute the intensity of feeling

is inversely proportional to the value

of the stakes at issue -- that is why

academic politics are so bitter.”

Wallace S. Sayre, 1905-1972

quoted in,

"Issawi's Laws of Social Motion" (1973)

3rd paradigm design based research
3rd ParadigmDesign-Based Research
  • Related to engineering and architectural research
  • Focuses on the design, construction, implementation and adoption of a learning initiative
  • Related to ‘Development Research’
  • Closest educators have to a “home grown” research methodology
design studies
Design Studies
  • iterative,
  • process focused,
  • interventionist,
  • collaborative,
  • multileveled,
  • utility oriented,
  • theory driven and generative
    • (Shavelson et al, 2003)
call centres innovation design based research case study
Call Centres Innovation Design-Based Research Case Study
  • Building Airplanes in the Air – EDS TV ad
Context: self paced study;

12,000 students in Business Undergraduate programs

Prior to 1995 traditional telephone/email and mail tutor model

Tutor Model

On call 2 hours/week

Part time, problem with knowledge of institution

knowledge of a single course

Personal relationship

Subject matter expertise

Personal knowledge base

Call Centre Model

Advisor on call 40 hours/week

Full time, steeped in university environment

All business curriculum

Customer relationship

Refers to academics for subject matter expertise

Formal FAQ and data collection


Call Centres:

Answer 80% of student inquiries

Saves over $100,000 /year

stage 1 informed exploration
Stage 1: Informed Exploration
  • Review of call centre literature
  • Interviews with current tutors and managers
  • Data collection on current processes and costs
  • Visit to other call centres, especially those in related but uncompetitive contexts
stage 2 enactment
Stage 2: Enactment
  • Design and coding using Lotus Notes
  • Project management, data collection on development problems and costs,
  • Pilot testing
  • Multiple iterations
stage 3 local evaluation
Stage 3 Local Evaluation
  • Interviews, focus groups with call centre staff
  • Student satisfaction surveys
  • Student interviews
  • Analysis of transaction logs and FAQ
  • Cost analysis
  • Interviews with tutors, union
stage 4 broader impact evaluation
Stage 4: Broader Impact Evaluation
  • Publishing of results
  • Trials in different contexts
    • AU General information
    • Computer help desk
  • Synthesis of application in different contexts
  • Adoption of Innovation framework
building the research culture
Building the Research Culture
  • Better dissemination within a combined research/practitioner communities
  • Better tools
  • More funding; less fighting
online journals
Online Journals
  • International Review of Research on Open and Distance Learning
research portals
Research Portals
  • International Research Foundation for Open Learning
    • Research for policy makers
    • Research for practitioners -
    • Support and training for researchers
    • Publishing research findings
research knowledge dissemination
Research Knowledge Dissemination
  • Need a robust, just in time, accessible anywhere system
  • Need for development of formal schema for an Educational Research Markup Language
  • Incentives for easy or automatic tagging
  • Is the Educational Semantic Web the Answer? See upcoming JIME special issue

Better distribution

Open Source Solution


4,200 downloads first two weeks

  • DE Research is grossly under-resourced to meet the magnitude of opportunity and demand
  • Paradigm wars are unproductive
  • Design-based research offers a promising new research design model
  • The semantic web offers promise of greatly enhanced collaboration and research dissemination

“Wisdom is fed equally from practical, theoretical, and ethical knowledge and from a large quantity of reflected experience”

Otto Peters, 2003 p. 137

your comments or questions most welcomed
Your Comments or QuestionsMost Welcomed !

Terry Anderson

quantitative research63
Quantitative research
  • Key words like “evidence based” “systematic review”scientific research”
  • Long tradition from the Natural sciences
  • “those who are seeking the strict way of truth should not trouble themselves about any object concerning which they cannot have a certainty equal to arithmetic or geometrical demonstration”
      • Rene Descartes 1596-1650
  • Since context is so pivotal in education, a great number of studies must be done to eliminate contextual variance and combined using meta analysis