1 / 12

Laboratory studies

Laboratory studies. Lorrie Faith Cranor September 2009. Laboratory study process. Design study Submit IRB application and wait for approval Recruit participants Screen participants Schedule participants Prepare for study Conduct study Instructions to participants Analyze data.

beulah
Download Presentation

Laboratory studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Laboratory studies Lorrie Faith Cranor September 2009

  2. Laboratory study process • Design study • Submit IRB application and wait for approval • Recruit participants • Screen participants • Schedule participants • Prepare for study • Conduct study • Instructions to participants • Analyze data

  3. Design study • See lecture on designing experiments • Some additional issues for lab studies • How to provide realistic setting and tasks in a lab environment • Whether or not to think aloud • How to observe, record • How much to pay • How long

  4. IRB application • Most usable security lab studies will be minimal risk and qualify for expedited review • However, use of deception may require more review • Use of deception must be justified by you and approved by IRB • Requires debriefing

  5. Recruit participants • To recruit students • Flyers around campus • Posts to campus email lists and bboards (misc.market) • CMU CBDR http://www.cbdr.cmu.edu/ • To recruit non-students • Flyers on bus stops, grocery stores, libraries, any local business that has a bulletin board that will let you post • Posts to Pittsburgh Craigslist • Asking your friends and neighbors • Distribute handouts or make announcements at local religious organizations, school parents’ organizations, athletic organizations, etc. • We’ve recruited families by handing out flyers at Schenley Oval during kids’ soccer games • We’ve recruited senior citizens by visiting senior citizen lunch at Squirrel Hill JCC • Many local organizations offer inexpensive ads in their newsletters (for example Colfax PTO http://colfaxpto.org/announcements/August2009adinfo.pdf • Some organizations may be willing to recruit members to participate as a fund raiser

  6. Screen participants • Decide what your criteria are to participate and develop screening survey to screen out those who don’t qualify • You may also want to use screening survey to help you balance demographics

  7. Schedule participants • Contact participants by email or phone to schedule their visit to the lab • CBDR includes an online scheduling system • Give participants clear directions for getting to the lab • Give them a way to contact you if they have to cancel or get lost finding the lab

  8. Prepare for study • Make sure you have copies of all study materials, consent form, payment record, etc. • Make sure all equipment is setup, working, has batteries, etc. Test it! • Develop a checklist of everything that needs to be setup before a participant comes into the lab so you can make sure you haven’t forgotten anything • Bring sufficient cash (or other payment) with you • Make sure you have exact change! • You can get reimbursed for human subjects payments, but if you plan ahead you can get a cash advance so you don’t have to layout money • Post signs so people can find the lab • Weekends and evenings you may have to meet people outside locked door • In some buildings posted signs routinely get taken down, plan to repost daily (or more frequently!) • If at all possible, have at least 2 people run study together • Useful to have someone greet participants and bring them to lab while someone else runs study • For some studies its useful to have someone ask questions and interact with participant (facilitator) while someone else takes notes • Practice, practice, practice!

  9. Instructions to participants • Describe the purpose of the evaluation • “I’m testing the product; I’m not testing you” • Tell them they can quit at any time • Demonstrate any equipment • Explain how to think aloud • tell us what they are trying to do • tell us questions that arise as they work • tell us things they read • Explain that you will not provide help • Explain the basic concept of the UI, but not too much • Can be done in writing or via video for consistency • Describe the task • give written instructions, one task at a time

  10. Study design problems • CookieMonster • https://addons.mozilla.org/en-US/firefox/addon/4703 • CookieSafe • https://addons.mozilla.org/en-US/firefox/addon/2497

  11. Discussion • Discussion about readings? • Questions about project?

More Related