security strategies for every stage of the testing process n.
Skip this Video
Download Presentation
Security Strategies for Every Stage of the Testing Process

Loading in 2 Seconds...

play fullscreen
1 / 49

Security Strategies for Every Stage of the Testing Process - PowerPoint PPT Presentation

  • Uploaded on

Security Strategies for Every Stage of the Testing Process. DeDe Hedlund , Creighton University Jeff Place, Questionmark Evangelist NCTA – Minneapolis, MN Friday Aug. 3, 2012, 11:30-12:30 pm. Goals of this Session. There are no shortage of test security challenges

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Security Strategies for Every Stage of the Testing Process' - ignatius-bates

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
security strategies for every stage of the testing process

Security Strategies for Every Stage of the Testing Process

DeDeHedlund, Creighton University

Jeff Place, Questionmark Evangelist

NCTA – Minneapolis, MN

Friday Aug. 3, 2012, 11:30-12:30 pm

goals of this session
Goals of this Session

There are no shortage of test security challenges

At each stage of the assessment process (Authoring, Scheduling, Delivery, and Reporting) solutions exist to meet security challenges

Let’s Identify the threats to watch out for

Show you today how certain product functionality can meet your test security needs by walking through a typical examination process and some real life examples



Content Theft



Fraud Triangle


  • Rationalization
  • Opportunity
  • Motivation
  • Impersonation
  • Content Theft
  • Cheating
low high to high high stakes tests
Low/High to High/High Stakes Tests

What’s at Stake?

Higher Stakes

Life and Limb

Promotion & Jobs & Legal Concern

Educational Exams


Elearning & Surveys

monitoring tests securely
Monitoring Tests Securely

ID Fraud

Protect Content

Minimizes Cheating





ID Fraud

Protect Content

Minimizes Cheating




authoring item banking and security
Authoring: Item Banking and Security
  • Security impacts
    • Validity of assessment results
    • Fairness to participants
  • One approach: security rights that are role-based
    • Set access permissions by role or profile
    • Limit an author’s or reviewer’s access to the various functions
    • Limit access to topic (item) folders
    • Limit what the author may do in those folders
  • Create participants and schedule assessment delivery
  • Manage participants individually or in hierarchical groups
  • Schedule access to assessments:
    • Limit dates
    • Limit attempts
    • Require proctor/invigilator
    • Deliver to certified test center
    • Online or offline delivery
  • Email Broadcast:Notifications to Participants
blended delivery
Blended Delivery

Levels of Monitoring

Stakes of Assessment

questionmark secure

Secure Browser

Regular Browser

Questionmark Secure
  • Stops people from printing questions
  • Stops people from typing in their own URL
  • Always display in full screen and it’s not possible to maximize or minimize
  • Stops people starting a new task
  • Does not provide menu options or icons
  • Disables control keys and task switching
  • Disables right-click menu options
  • Disables the ability to start new applications
  • Prevents going backwards to a previous page
  • Stops people exiting in a high stakes, proctored, environment
  • Hides the HTML source
  • Provides an API to control certain functions of a browser from the server
  • Server can command Questionmark Secure to display a toolbar
browser check
Browser Check

Ensures Test Takersuse a compatible browser

delivery security
Delivery Security
  • Perception can be configured to use SSL to ensure communication between participants and the server are secure
  • Data in item repositories are encrypted (QML is encrypted) providing added item bank security
  • All participant scoring is conducted on the Perception server (rather than on client/participant PCs)
  • Scoring algorithms are not sent in any way to participant PCs

Limiting Item Exposure

  • Question by Question (QxQ) delivery options limit the amount of content shown at one time
  • Randomization of questions and choices to prevent cheating
  • Score List Report
    • IP addresses (as expected?)
    • Time taken (too quick, too long?)
    • Easy flagging of participants who get perfect scores for further investigation
  • Coaching Report
    • Drill down on an individual participant, comparison to benchmark, use in appeal process
  • Item Analysis Report
    • Determine psychometric performance and examine item difficulty drift over time
  • Test Analysis Report
    • Assessment defensibility review and test difficulty drift
case studies
Case studies

Case studies from Creighton University

  • What their test security needs were
  • How our software addressed those needs
  • Demo of solutions
creighton university school of pharmacy health professions
Creighton University School of Pharmacy & Health Professions
  • Private Jesuit University located in Omaha, NE
  • 2001 started the first and only accredited truly distance based full time Doctor of Pharmacy program
  • Issues to resolve and security requirements
    • How to securely deliver and administer exams to students located throughout the country
    • Paper and pencil were too expensive and it’s difficult to distribute and collect approximately 5600 exams per semester
    • Control who sees what test and when
    • Lock down the computer to eliminate printing, instant messaging, browsing the internet during the tests
    • Ability for students to “mark” an identified area of a graphic
creighton university school of pharmacy health professions1
Creighton University School of Pharmacy & Health Professions

Solutions offered to meet requirements:

  • Provided online testing solution and eliminated almost all paper and pencil tests to reduce costs
  • 22 Question types including Hotspot allowing online exams to replicate past paper exam formats with the additional ability to provide instant feedback.
  • Ability to randomly deliver different variations of an exam
  • Manage large item banks of questions including security for collaborative authoring among faculty
  • Lock down browser
  • Scheduling (right test to right student)


  • Hotspot question type & feedback text/images
  • Password system & variations of an exam
  • Item Banking and security for collaborative authoring
  • Questionmark Secure
  • Scheduling
creighton university school of pharmacy health professions2
Creighton University School of Pharmacy & Health Professions

Additional security measures

  • Established guidelines for choosing new proctor sites
  • Formalized the process of communicating what is expected from a proctor site
  • Proctor Certification forms are required from every student who completes an exam
  • Exam specific instructions are emailed to the proctors
remote video monitoring
Remote Video Monitoring



Laptops and PCs

candidate participant logs in
Candidate (Participant) Logs in

As Software Secure and Questionmark integration is for higher stakes exams a Participant will be referred to as a Candidate

schedule appointment exam
Schedule Appointment Exam

Candidate selects exam and “Schedules Appointment”

Exams (4)

Surveys (1)

review schedule
Review Schedule

If the exam is to be schedule in the next 3 days or if the participant might be expected to pay some of the cost of the exam then there are more steps

selects exam to take
Selects Exam to Take

Exams (4)

Surveys (1)

proctoru software secure
ProctorU / Software Secure
  • Camera and Microphone Access
  • Identity Checks
  • Captures Photo, Proctor Checks against files
  • Proctor checks has Participant scan the room
security checks complete candidate takes exam
Security Checks Complete Candidate Takes Exam
  • After Authentication Candidate takes Exam
  • Video and screens are monitored during exam
mobile test centers
Mobile Test Centers
  • Use notebooks/3G laptops/iPads to create mobile test centers
  • Enables greater flexibility in where you deliver assessments
  • Assessments can be conducted in ‘on location’ environments

Slide 10

securing an ipad
“Securing” an iPad…
  • An app should prevent candidates from easily getting to other URLs
  • But – access to “home button” allows task-switching
  • Hardware solutions are available to enable use of iPad in a “kiosk” mode

Slide 11

  • The higher the stakes, the higher the propensity to cheat –still require human monitoring!
  • Bandwidth – Be sure to test the wifi and/or 3G signal strength prior to event

Slide 12

considerations for cheat resistant questions and assessments
Considerations for cheat-resistant questions and assessments


Refresh items regularly

Build large item banks

Track item parameters

Tests that are unique for each test taker make it hard to share answers

Adaptive tests provide different questions based on responses to previous questions

improve your multiple choice questions
Improve Your Multiple Choice Questions

Answer choices should be roughly the same length and kept as short as possible.

Provide a minimum of three answer choices and a maximum of five. Four is considered optimal.

Keep your writing clear and concise – you’re testing knowledge, not reading comprehension.

Make sure that you’re putting the correct answer in the first two positions as often as the last two positions.

extended matching question type
Extended Matching Question Type
  • The number of answer options depends on the logical number of realistic options for the test taker.)
  • The same answer choice could be correct for more than one question in the set
  • Some answer choices may not be the correct answer for any of the questions
  • So it is difficult the answer this type of question correctly by chance.
  • A well-written lead-in question is so specific that students understand what kind of response is expected, without needing to look at the answer options.
closing and questions
Closing and questions
  • Thank you for your time and attention!
  • Questions/Comments?
  • Visit our booth to talk to us more
  • We have a whitepaper on this topic (Delivering Computerized Assessments Safely and Securely)
    • Visit our website to download:
thank you for attending

Thank you for attending!


Jeff Place