TESTING SERVICES - TESTING Security
Through our partnership with Caveon Test Security we offer a full suite of test security technologies and service including:
Caveon Security Audits(TM)
We use a combination of qualitative and quantitative methods to analyze a test program's existing security practices, its state of test security, and to produce recommendations for security enhancement.
We review policies and procedures for managing test item and other secure materials as they are maintained and communicated among a test program's staff, vendors, and various collaborators in the test development, administration, scoring and reporting, and item pool maintenance processes. This is accomplished through review of the test program's existing documentation and structured interviews with key test program staff.
To provide some measure of the effectiveness of existing practices, we will also analyze test results for indications of test fraud and theft, and perform a comprehensive investigation of the Internet for unauthorized disclosures of protected content associated with the test program.
Caveon Data Forensics(TM)
Caveon's patent-pending Data Forensics system utilizes sophisticated, statistical analyses of test-response data to identify patterns indicative of test fraud, including cheating and piracy. In addition to its sophisticated analysis of item responses and, for test administrations that are delivered via computer, latencies for "aberrance", Data Forensics examines the responses from all possible pairs of examinees within and across test administration centers to assess the extent to which examinee response patterns are so similar that they may be indicative of organized cheating, or collusion (e.g., use of common "test prep" materials obtained from a braindump site or test prep course). This addresses a test program's need to identify and remove test content from operational use due to changes in their performance that indicate that the materials may no longer be secure.
Data Forensics also analyzes temporal characteristics of the tests as a means of ensuring the test performance is stable and within operational parameters. These temporal analyses are conducted at the test score level using Caveon's proprietary change detection algorithms to separate the signal from the noise. Test score and test aberrance statistics are tracked simultaneously to determine whether test score changes are induced by changes in test aberrance or are occurring independently. Additionally, this temporal analysis depicts trends in test score changes.
When supplemented by comprehensive analysis of test retake policy violations and score volatility between retakes, the resulting Data Forensics reports provide clients with critical information regarding where and when the suspected activity occurred, by whom, and its effects on the health of the client's testing program.
Caveon Web Patrol(TM)
Caveon recommends a Web Patrol one-year subscription to detect the level of risk and exposure of a test program's test items. The Caveon Web Patrol services search the World Wide Web to locate web sites that represent a risk to the security to a test program. Such sites might include brain-dump sites, sites that offer test preparation training/education such that there is a risk of actual (operational) test questions being used in the training, E-Bay, and other forums in which actual test items may be revealed or proxy test takers offer their services. Caveon Web Patrol does not search for or identify chat rooms or listservs that may be a security risk, but if such sites are discovered, they may be probed periodically and their URL's provided in periodic reports.
The proposed level of service provides a test program with comprehensive searches of the web to determine the level of current risk of exposed test items or other related materials that may put a program's tests at risk. We recommend searches at least six times during the year and, for paper-and-pencil tests at least four times a year or time periods corresponding with key periodic assessments. For computer-based tests that are offered on a continuous testing schedule, more frequent Web Patrols may be appropriate.
Caveon Web Patrol is a continual process that provides for regular reports after each search, with intervening reports made available if high-risk sites are identified in the intervals between regular reports.
Because the Web Patrol is comprehensive, it produces "shared efficiencies" between Caveon's clients. When a site is located and rated for one client, it can be immediately reevaluated and reported to others. In one instance, Caveon was able to instantly generate a list of high-risk sites for one client based upon the work performed for another. This cross-fertilization benefits Caveon's clients and the security of the testing industry as a whole.
Test Development
Free Test Security Evaluation
Please send your questions or requests to info@kryteriononline.com. A business development manager will contact you immediately to answer your questions and provide additional information.
If you prefer, you can call our business development department at 866.579.8374 x4687. We can schedule an online demo at a convenient time for you and your team.
Download the print version. (172K PDF)