If you’d like to review the previous articles in this series, which were posted back in November, you can find them here: Part 1: Complaints & Appeals Related to Testing: An Overview and Part 2: Considering Your Appeals Process.

This is the third article in this series on complaints and appeals and it is intended to give courage and hope to some of you in the HR profession who are dealing with rules governing complaints and appeals that do not support sound test development and validation procedures. If we are to support and improve the effectiveness of testing and the value of the work done by those in our profession, we need to recognize that there are times we need to work to change rules that are contrary to sound practices. While being a change agent can be fraught with risks, it can also produce rewards. Before going forward with any efforts to modify existing rules, it is critical to assess the climate in which you work and the impact appeal procedures have on the utility of the tests you are using.

Some of the basic things we know about test development and test validation include the fact that tests only measure the KSAP’s (knowledge, skills, abilities and personal characteristics) that an individual possesses at the time of testing. We also know that most tests we use in Human Resources are either aptitude tests or achievement tests.

In general terms:

• Aptitude tests measure one’s ability to learn and retain information over time and they are usually the types of test used for entry-level testing.
• Achievement tests are designed to measure one’s knowledge of a particular subject after having received training and/or experience in that area.

Anything that occurs post test in regard to providing candidates the opportunity to review the test and appeal test items changes candidates’ body of knowledge that can be applied to the test and therefore negatively impacts the reliability of the test. That is, we are now measuring candidates’ abilities to conduct research and make cogent arguments regarding the quality of test items and their answers compared to keyed answers. We are no longer able to determine what candidates knew or did not know at the time of the test. So when we change scores for candidates based on appeals we are giving them credit for information they may or may not have had during the test. That means we are no longer measuring what the test was intended to measure and alterations in scores that negatively impact reliability also reduce the validity of the test. (more…)

As stated in the last article, appeals are typically more formal than complaints and there are usually written rules and procedures that govern the handling of appeals. These rules typically spell out how an appellant will go about appealing and how the agency will go about responding to the appeal.

The related rules range from general to very specific and they often can be found written out in the civil service rules for the agency or in a separate document of their own. At times, unions negotiate appeal procedures, but they are not typically part of mandatory subjects of bargaining. At one point in my career, I worked for a police agency that had been rocked by a cheating scandal. To prevent further cheating they created a very onerous and expensive test creation and appeal process that reeked havoc on the ability to develop valid and reliable written tests.

This brings me back to a point I made in the previous article and I will stress again here. If you have written appeal procedures that are not consistent with sound test development and validation strategies, you should consider it a goal to get them rewritten and approved so they do. Truly your rules should work for your test development and validation program and not against it. (more…)

If you have an active selection program that processes large numbers of candidates through successive hurdles type of processes that include written and oral exams, the probability you are going to get complaints and appeals is high. The number you receive and how you handle them is largely within your control. While it is true that most jurisdictions have an appeal process that is spelled out by their Civil Service Rules or other regulations that guide their operations, it is also generally true that these rules are subject to change. So I acknowledge that, for the present time, the manner in which you deal with complaints and appeals may be dictated to you. I would also like to stress that in your role as an HR Professional it is part of your responsibility to do what you can to ensure that your rules reflect current practices and procedures in the field of testing.

You may also have written contracts with unions and/or consultants and test publishers that specify how appeals will be handled and in those cases you are obligated to follow those guidelines during the life of the contracts. However, I still believe the number of complaints and appeals you receive can be minimized by your approach to the testing process. Unfortunately, even though a lot of the finesse involved in handling applicants and their issues must be learned through experience, having good guidelines to follow and a positive customer service attitude can go a long way in mitigating the impact of complaints and appeals. (more…)