Test Security and Cheating: Part 1

If your jurisdiction has ever been the target of a cheating scheme or scandal then you are very familiar with the costs of cheating. Writing new tests, conducting new recruitments and administering new selection procedures are time-consuming and costly, yet they represent only a portion of what could be considered the costs of cheating. Perhaps the biggest potential costs of cheating are those incidents where cheating goes undetected, resulting in incorrect selections for promotions or filling entry-level positions. Whatever the costs and whatever the long-term impacts may be, the sad truth is that cheating still occurs.

A cheating incident from 2009 underscores the impact of cheating scandals on agencies and should serve as a warning for individuals with the responsibility of preventing cheating, as well as any individual that may consider cheating. In this case, a member of the testing committee provided test questions to a lieutenant who subsequently provided them to an officer studying for the test. That officer went to internal affairs and an investigation was launched. In addition to scrapping the test,  four police officials were temporarily removed from their duties pending the conclusion of the investigation. Two eventually left the department. The cloud of suspicion and distrust continues to hang over the department and the ultimate negative affects are incalculable.

As stated before, incidents like these should serve as lessons to those on both sides of the situation. Test users can always do more in terms of providing security for their tests and cheaters rarely succeed, ultimately paying the highest price of all involved.

Unfortunately, I had the opportunity to witness the impact of cheating first hand when one of the jurisdictions I worked for in the past was rocked by a scandal involving a wrongful promotion and subsequent misbehavior on the part of the individual that was promoted. Not only was the department liable for the actions of the individual that had been promoted to sergeant, but it also suffered incalculable consequences in regard to those who were not promoted and became disenchanted with serving this particular department.

Among the many lessons learned in this experience was one I see many practitioners failing to learn; that complacency is a cheater’s biggest ally. While it is critical to follow all the known test security and anti-cheating measures that are available in the literature, and through common sense as well as written test security agreements, it is also critical to learn from the mistakes and experiences of others. Therefore, I think it is valuable to relate the circumstances surrounding the cheating incident I was involved in uncovering so I can share the lessons that I learned and provide food for thought for individual readers in terms of how their agencies may be vulnerable to cheating.

In that regard, IPMA-HR has had the foresight to develop very strict item challenge procedures for promotional tests that include not allowing candidates to take any test materials from the testing site or any item challenges meetings held within the department.

In particular, the police agency that I worked for had experienced an incident of cheating before my tenure there and they had an elaborate test review and item challenge procedure. The regulations that they promulgated to prevent cheating and provide a “fair” challenge system created the perfect situation for the potential for cheating. Unfortunately, this wasn’t truly seen until the incident occurred and in that regard, we had overlooked the potential for a group of individuals working together to pull off an intricate scheme. So the lessons learned included being aware that cheaters do not always act alone and that plans for cheating can be elaborate. Hopefully the details of what occurred will provide insights regarding things you or your jurisdiction might have overlooked in your test security.

In this particular case, the jurisdiction review process involved providing each candidate with a list of the items he missed along with the correct or keyed answer immediately after completing the test. This was done to allow each candidate the opportunity to compare his answers with keyed answers and determine if he wanted to challenge an item. Challenges could take the form of asserting that their answer was as good as or better than the keyed answer or that the item itself was flawed and should be removed from the test. The challenge process was as competitive as the rest of the selection process, with test participants being aware that the outcome of appeals could impact their ultimate ranking on the certification list. In particular, candidates with the lowest scores had the most to gain by appealing items and winning their arguments so the system encouraged them to file numerous appeals regardless of their merit.

The test was administered following standard protocol with the test session being limited to test takers and proctors. Each candidate signed in with picture ID and was given one test booklet and a Scantron answer sheet, both of which were numbered. Candidates were allowed as much time as they wanted to complete the test and were escorted to the restroom as needed, but were not allowed to the leave the test room for any other reason. Upon completing the exam, each candidate was given a computer printout listing the number of the items he had missed along with his answer and the keyed answer.

There were two unusual occurrences during the test. Within the first twenty minutes of the exam, one officer, who had a history of discontent with the department, stood up and said something to the effect that he’d “had enough of this B.S.”  and left the room. He proceeded to take his answer sheet to the scoring table, “just for the heck of it,” as he announced.

The second unusual occurrence was that a sergeant that was on duty stopped by in his police unit to “see how the test was going.” He was told that he was not permitted to be at the test site so after agreeing to leave he asked if he could use the rest room which he was permitted to do and then he left.

It was not until the end of the test and all candidates were finished that a third piece of information was provided that put the other two occurrences in a different light. One of the proctors informed HR Staff that he had accompanied one particular candidate to the restroom more than ten times during the two hours he spent working on the test. At that point, considering that the sergeant that had dropped by, the officer who finished early, and the officer who made excessive trips to the restroom were all on the same squad and known to be very close, we were able to put together a theory of what happened.

The candidate that finished the test first left the test site with an almost complete key to the test since he had done so poorly and had been provided with all the correct answers to the items he missed. Then he met up with the on duty sergeant to provide him with the key. The sergeant, per a prearranged agreement, went to the test site and deposited a crib sheet in the toilet paper dispenser in a predetermined stall in the rest room. This happened to be stall four — the same stall the candidate that was the focus of our investigation used every time he went in the restroom.

As part of our investigation, we reviewed the answer sheets provided by all candidates. We determined that the average number of erasures and item changes per answer sheet was approximately three, while the candidate in question had over eighty. In addition, about fifty percent of the time when answers were changed, the erasure marks indicated that the candidate had changed the answer from wrong to right and about fifty percent of the time the candidate changed from right to wrong. The answer sheet competed by the candidate under suspicion indicated that among his erasures, he had gone from wrong to right in every change he made except one. That meant that some how he gained an insight that allowed him to change his original wrong answer to a correct answer 98% of the time. So in addition to the other suspicious behavior, this candidate defied all odds and was able to correct nearly every mistake he had originally made.

While HR believed we had solid proof of what had happened and how, the Sheriff in office at the time chose not to take any action, believing that the evidence would not withstand a challenge by the union. Instead, the list was certified without challenge and the individual who had cheated was promoted, only to be terminated at a later date for use of excessive force that was caught on tape.

While the next article will delineate a number of precautions that should be taken regarding test security, along with several steps involved in creating an anti cheating environment, it is hoped that this particular story of cheating will encourage you to take a hard look at the procedures you have in place and identify areas that may make your jurisdiction vulnerable. HR needs to be as creative in prevention procedures as those that would cheat us are in developing their schemes. Knowing what has been attempted before is a critical way of recognizing and stopping schemes before they happen.

2 thoughts on “Test Security and Cheating: Part 1

  1. Pingback: Test Security and Cheating: Part 2 | Assessment Services Review

  2. Pingback: Help Us Blow Out Our Birthday Candle as the ASR Turns One! | Assessment Services Review

Leave a Reply