Security strategies for every stage of the testing process
This presentation is the property of its rightful owner.
Sponsored Links
1 / 49

Security Strategies for Every Stage of the Testing Process PowerPoint PPT Presentation


  • 65 Views
  • Uploaded on
  • Presentation posted in: General

Security Strategies for Every Stage of the Testing Process. DeDe Hedlund , Creighton University Jeff Place, Questionmark Evangelist NCTA – Minneapolis, MN Friday Aug. 3, 2012, 11:30-12:30 pm. Goals of this Session. There are no shortage of test security challenges

Download Presentation

Security Strategies for Every Stage of the Testing Process

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Security strategies for every stage of the testing process

Security Strategies for Every Stage of the Testing Process

DeDeHedlund, Creighton University

Jeff Place, Questionmark Evangelist

NCTA – Minneapolis, MN

Friday Aug. 3, 2012, 11:30-12:30 pm


Goals of this session

Goals of this Session

There are no shortage of test security challenges

At each stage of the assessment process (Authoring, Scheduling, Delivery, and Reporting) solutions exist to meet security challenges

Let’s Identify the threats to watch out for

Show you today how certain product functionality can meet your test security needs by walking through a typical examination process and some real life examples


Threats

Threats

Impersonation

Content Theft

Cheating


Issues

Issues

Fraud Triangle

Threats

  • Rationalization

  • Opportunity

  • Motivation

  • Impersonation

  • Content Theft

  • Cheating


Low high to high high stakes tests

Low/High to High/High Stakes Tests

What’s at Stake?

Higher Stakes

Life and Limb

Promotion & Jobs & Legal Concern

Educational Exams

Tests

Elearning & Surveys


Monitoring tests securely

Monitoring Tests Securely

ID Fraud

Protect Content

Minimizes Cheating

X

X

X


Providers

Providers

ID Fraud

Protect Content

Minimizes Cheating

X

X

X


Assessment management process

Assessment Management Process


Authoring item banking and security

Authoring: Item Banking and Security

  • Security impacts

    • Validity of assessment results

    • Fairness to participants

  • One approach: security rights that are role-based

    • Set access permissions by role or profile

    • Limit an author’s or reviewer’s access to the various functions

    • Limit access to topic (item) folders

    • Limit what the author may do in those folders


Scheduling

Scheduling

  • Create participants and schedule assessment delivery

  • Manage participants individually or in hierarchical groups

  • Schedule access to assessments:

    • Limit dates

    • Limit attempts

    • Require proctor/invigilator

    • Deliver to certified test center

    • Online or offline delivery

  • Email Broadcast:Notifications to Participants


Blended delivery

Blended Delivery

Levels of Monitoring

Stakes of Assessment


Questionmark secure

Secure Browser

Regular Browser

Questionmark Secure

  • Stops people from printing questions

  • Stops people from typing in their own URL

  • Always display in full screen and it’s not possible to maximize or minimize

  • Stops people starting a new task

  • Does not provide menu options or icons

  • Disables control keys and task switching

  • Disables right-click menu options

  • Disables the ability to start new applications

  • Prevents going backwards to a previous page

  • Stops people exiting in a high stakes, proctored, environment

  • Hides the HTML source

  • Provides an API to control certain functions of a browser from the server

  • Server can command Questionmark Secure to display a toolbar


Browser check

Browser Check

Ensures Test Takersuse a compatible browser


Delivery security

Delivery Security

  • Perception can be configured to use SSL to ensure communication between participants and the server are secure

  • Data in item repositories are encrypted (QML is encrypted) providing added item bank security

  • All participant scoring is conducted on the Perception server (rather than on client/participant PCs)

  • Scoring algorithms are not sent in any way to participant PCs


Delivery security features

Delivery Security Features


Security strategies for every stage of the testing process

Limiting Item Exposure

  • Question by Question (QxQ) delivery options limit the amount of content shown at one time

  • Randomization of questions and choices to prevent cheating


Reporting

Reporting

  • Score List Report

    • IP addresses (as expected?)

    • Time taken (too quick, too long?)

    • Easy flagging of participants who get perfect scores for further investigation

  • Coaching Report

    • Drill down on an individual participant, comparison to benchmark, use in appeal process

  • Item Analysis Report

    • Determine psychometric performance and examine item difficulty drift over time

  • Test Analysis Report

    • Assessment defensibility review and test difficulty drift


Case studies

Case studies

Case studies from Creighton University

  • What their test security needs were

  • How our software addressed those needs

  • Demo of solutions


Creighton university school of pharmacy health professions

Creighton University School of Pharmacy & Health Professions

  • Private Jesuit University located in Omaha, NE

  • 2001 started the first and only accredited truly distance based full time Doctor of Pharmacy program

  • Issues to resolve and security requirements

    • How to securely deliver and administer exams to students located throughout the country

    • Paper and pencil were too expensive and it’s difficult to distribute and collect approximately 5600 exams per semester

    • Control who sees what test and when

    • Lock down the computer to eliminate printing, instant messaging, browsing the internet during the tests

    • Ability for students to “mark” an identified area of a graphic


Creighton university school of pharmacy health professions1

Creighton University School of Pharmacy & Health Professions

Solutions offered to meet requirements:

  • Provided online testing solution and eliminated almost all paper and pencil tests to reduce costs

  • 22 Question types including Hotspot allowing online exams to replicate past paper exam formats with the additional ability to provide instant feedback.

  • Ability to randomly deliver different variations of an exam

  • Manage large item banks of questions including security for collaborative authoring among faculty

  • Lock down browser

  • Scheduling (right test to right student)

    Demos

  • Hotspot question type & feedback text/images

  • Password system & variations of an exam

  • Item Banking and security for collaborative authoring

  • Questionmark Secure

  • Scheduling


Demonstration

Demonstration


Variations of an exam

Variations of an Exam


Item banking

Item Banking


Creighton university school of pharmacy health professions2

Creighton University School of Pharmacy & Health Professions

Additional security measures

  • Established guidelines for choosing new proctor sites

  • Formalized the process of communicating what is expected from a proctor site

  • Proctor Certification forms are required from every student who completes an exam

  • Exam specific instructions are emailed to the proctors


Remote video monitoring

Remote Video Monitoring

Custom

Hardware

Laptops and PCs


Candidate participant logs in

Candidate (Participant) Logs in

As Software Secure and Questionmark integration is for higher stakes exams a Participant will be referred to as a Candidate


Schedule appointment exam

Schedule Appointment Exam

Candidate selects exam and “Schedules Appointment”

Exams (4)

Surveys (1)


Schedule day and time

Schedule Day and Time


Review schedule

Review Schedule

If the exam is to be schedule in the next 3 days or if the participant might be expected to pay some of the cost of the exam then there are more steps


Payment process

Payment Process


Candidate takes exam when it was scheduled

Candidate Takes Exam When It Was Scheduled


Candidate participant logs in1

Candidate (Participant) Logs in


Selects exam to take

Selects Exam to Take

Exams (4)

Surveys (1)


Proctoru software secure

ProctorU / Software Secure

  • Camera and Microphone Access

  • Identity Checks

  • Captures Photo, Proctor Checks against files

  • Proctor checks has Participant scan the room


Security checks complete candidate takes exam

Security Checks Complete Candidate Takes Exam

  • After Authentication Candidate takes Exam

  • Video and screens are monitored during exam


Potential applications for mobile delivery

Potential Applications for Mobile Delivery


Mobile test centers

Mobile Test Centers

  • Use notebooks/3G laptops/iPads to create mobile test centers

  • Enables greater flexibility in where you deliver assessments

  • Assessments can be conducted in ‘on location’ environments

Slide 10


Securing an ipad

“Securing” an iPad…

  • An app should prevent candidates from easily getting to other URLs

  • But – access to “home button” allows task-switching

  • Hardware solutions are available to enable use of iPad in a “kiosk” mode

Slide 11


Considerations

Considerations:

  • The higher the stakes, the higher the propensity to cheat –still require human monitoring!

  • Bandwidth – Be sure to test the wifi and/or 3G signal strength prior to event

Slide 12


C heat resistant questions and assessments

Cheat-resistant questions and assessments


Considerations for cheat resistant questions and assessments

Considerations for cheat-resistant questions and assessments

Randomization

Refresh items regularly

Build large item banks

Track item parameters

Tests that are unique for each test taker make it hard to share answers

Adaptive tests provide different questions based on responses to previous questions


Improve your multiple choice questions

Improve Your Multiple Choice Questions

Answer choices should be roughly the same length and kept as short as possible.

Provide a minimum of three answer choices and a maximum of five. Four is considered optimal.

Keep your writing clear and concise – you’re testing knowledge, not reading comprehension.

Make sure that you’re putting the correct answer in the first two positions as often as the last two positions.


Avoid previously delivered questions

Avoid Previously Delivered Questions


Extended matching question type

Extended Matching Question Type

  • The number of answer options depends on the logical number of realistic options for the test taker.)

  • The same answer choice could be correct for more than one question in the set

  • Some answer choices may not be the correct answer for any of the questions

  • So it is difficult the answer this type of question correctly by chance.

  • A well-written lead-in question is so specific that students understand what kind of response is expected, without needing to look at the answer options.


Scenario based questions

Scenario Based Questions


Scan a qr code with your mobile to start an assessment

Scan a QR Code with your mobile to start an assessment!


Closing and questions

Closing and questions

  • Thank you for your time and attention!

  • Questions/Comments?

  • Visit our booth to talk to us more

  • We have a whitepaper on this topic (Delivering Computerized Assessments Safely and Securely)

    • Visit our website to download: http://www.questionmark.com/us/whitepapers/index.aspx


Thank you for attending

Thank you for attending!

DeDeHedlund

[email protected]

Jeff Place

[email protected]


  • Login