rapid research in the educational technology development process l.
Skip this Video
Loading SlideShow in 5 Seconds..
Rapid Research in the Educational Technology Development Process PowerPoint Presentation
Download Presentation
Rapid Research in the Educational Technology Development Process

Loading in 2 Seconds...

play fullscreen
1 / 34

Rapid Research in the Educational Technology Development Process - PowerPoint PPT Presentation

  • Uploaded on

Rapid Research in the Educational Technology Development Process Cara Lane , PhD Research Scientist Catalyst Research & Development Office of Learning & Scholarly Technologies UW Technology University of Washington Session Outline Part I: Research overview

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Rapid Research in the Educational Technology Development Process' - Ava

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
rapid research in the educational technology development process

Rapid Research in the Educational Technology Development Process

Cara Lane, PhD

Research Scientist

Catalyst Research & Development

Office of Learning & Scholarly Technologies

UW Technology

University of Washington

session outline
Session Outline

Part I: Research overview

Part II: Needs assessment & usability

Part III: Best practices & institutional data

our work
Our Work
  • Web Tools
    • 13 modular tools
    • Free to UW community
  • Learning Spaces
    • Student computer labs
    • Computer classrooms
    • Advanced facilities
  • Knowledge Services
    • Center for Teaching, Learning, & Technology
    • Faculty and student workshops
    • Research
types of research
Types of Research
  • Assessing user needs
  • Conducting usability studies
  • Identifying best practices
  • Gathering institutional data
approaching research as service
Approaching Research as Service
  • Our research team
    • 1.5 staff members and 1 graduate assistant
    • Collaboration with designers, outreach staff, and anyone else we can find
  • Process:
    • Unlearning what we learned about research in graduate school
    • Overcoming unique challenges
    • Developing new practices
unlearning research myths
Unlearning Research Myths

Myth #1:

All research should be grounded in theory

Myth #2:

All research must present your best thinking

Myth #3:

Research is an independent or small team endeavor

research challenges
Research Challenges
  • Frequent organizational change
  • Fitting research into development timeline
  • Does UW really want to learn best practices from the IT shop?
  • Perception of bias
  • Where does this type of research fit within your institution?
  • How do you currently gather information about technology use and needs?
  • Are there projects you are currently working on where it would be useful to add a research component?
part ii needs assessment usability studies or how fast do you need that
Part II:

Needs Assessment & Usability Studies


How fast do you need that?

needs assessment
Needs Assessment
  • CommonView
    • Course or project workspaces
    • Integrate multiple Web tools; maintain flexibility of use
  • Questions
    • How do Catalyst users integrate multiple tools?
    • How do users of courseware use features of those tools?
    • What are the unmet needs of both groups?
challenge time
Challenge: Time
  • Pace of development cycle
    • 1 month to develop study, recruit participants, and gather data
  • Our response:
    • Collected internal evaluation data only
    • Expanded team: designers & outreach staff
    • Developed new focus group procedures
    • Streamlined data analysis
focus group format
Focus Group Format
  • Traditional model
    • IRB approval for research study
    • Formal, written consent procedures
    • Audio-recording and transcription of session
  • New procedures
    • Evaluation data for UW internal use
    • No identifying information recorded
    • 1-2 note-takers per session
    • Observations added to notes/gaps filled
data analysis process
Data Analysis Process
  • By population
  • Meeting of leaders and note-takers
  • Review of notes
  • Documentation of findings
    • Most students reported accessing their course Web sites using bookmarks. They had difficulty finding the sites from computers other than their own.
challenge bias
Challenge : Bias
  • UW committees want to know user needs
  • We have the most information
  • Can others trust our information?
  • Our response:
    • Share overall summary of findings (incl. those unflattering to our group)
    • Detail purpose of study and limitations of findings
    • Explain research and development process
usability studies
Usability Studies
  • Conducted by design team
  • Several times during design and development cycle
  • Challenge:
    • Quickly finding sufficient numbers of diverse users to test tools
  • Our response:
    • Creation of Catalyst User Experience (CUE) project
  • Began in 2007
  • 241 members
  • Initial membership survey
  • 11 targeted usability studies and surveys
    • CUE: 60-90% response to invitations
    • Old methods: 5-20% response
  • Biannual newsletter
  • Examples of other fast-paced research techniques?
  • What strategies do you use to share information about user needs among IT groups and with other segments of university of college community?
part iii best practices institutional data or who wants to learn this stuff from the it shop
Part III:

Best Practices & Institutional Data


Who wants to learn this stuff from the IT shop?

identifying best practices
Identifying Best Practices
  • Inter/National Coalition for Electronic Portfolio Research (cohort 1)
  • 3-year study of ePortfolio use
    • UW contests
      • Identify spectrum of student use
      • Spring 2005 & 2007
    • Case study: Expository Writing Program
      • Gather knowledge about implementation path
      • 2005/6 pilot & 2006/7 option
our process
Our Process
  • IRB: Minimal risk research
  • Data Gathered:
    • Interview transcripts for students, TAs, and administrators
    • Surveys of contest applications, EWP students, and EWP TAs
    • Review of student portfolios and instructional materials
challenge getting campus to listen
Challenge: Getting campus to listen
  • Are we the right people to be discussing best practices?
  • Is pedagogy our focus right now?
  • Our response:
    • Practical reports for UW audience
    • Breadth of data; can tell multiple stories
    • Academic publications for larger audience
gathering institutional data
Gathering Institutional Data
  • 2008 Faculty, Teaching Assistant, and Student Surveys on Learning and Scholarly Technologies
  • Campus-wide surveys
    • All instructors of record (approx. 4000)
    • Stratified sample (by broad disciplinary group) of 1000 TAs
    • Stratified sample of 5000 students
challenge time30
Challenge: Time
  • How can we gather enough information to write up-to-date surveys when the survey itself is our main vehicle for gathering information on current needs?
  • Our Response:
    • Autumn focus groups with faculty, TAs, students
    • IRB: Exempt research
    • Utilized needs assessment focus group process
    • Expanded team: staff from UW Technology, Libraries, and Office of Educational Assessment
    • Provided focus group training sessions for staff
challenge getting campus to listen32
Challenge: Getting campus to listen
  • How can the survey results best inform technology-related decisions on campus?
  • Our response:
    • Survey focus on goals and opinions, rather than simple snapshot of use
    • Extensive collaboration
    • Units who can implement changes are part of team
    • Practical reports and presentations for UW audience
    • Academic publications for larger audience
  • Examples of other similar projects or challenges?
  • What strategies can we try for establishing research partnerships across campus?
  • What strategies can we use to share research processes and findings between institutions?
contact information
Contact Information
  • Cara Lanecgiacomi@u.washington.edu
  • Catalysthttp://catalyst.washington.edu