Medical doctors and psychologists rarely rely on the results of one clinical test when making diagnoses. Similarly, selection experts recommend using a battery of selection instruments when making employee selections. The people in these professions realize that the accuracy and reliability of their conclusions are greatly enhanced when they have a broader range of information on those being evaluated.
In selection it is often critical to measure quite divergent knowledge, skills, and abilities, which necessitates the use of multiple selection instruments as part of a battery that comprises the selection process. Most jobs require cognitive abilities and some require a body of knowledge, which in many cases can be measured by a written exam. In addition, most jobs require some degree of ability to communicate verbally. Since written tests can not measure verbal communication, a second test, usually a structured interview is necessary to measure whether or not a candidate possess the verbal abilities required for the target job.
In addition to measuring these abilities in candidates, many positions require additional abilities which require the use of additional selection instruments. Many classes like police officer, fire fighter, corrections officer and park ranger require the measurement of candidates’ physical abilities, psychological stability, medical fitness and suitability of background. To utilize these instruments effectively and efficiently, they must be combined in a manner that provides the greatest support for administration of the selection process and maximization of each instrument’s validity. Combining the information from multiple instruments is where the employee selection model differs from the medical model.
Two critical concepts in the combining process involved in selection are what are termed “successive hurdles” and “weighting.” Both terms are pretty much what they sound like they should be. “Successive hurdles” refers to the sequencing of the administration of the instruments in the selection plan with candidates being required to “jump” each hurdle. “Weighting” refers to the portion or percentage an instrument contributes to the final score for candidates. Weighting is typically established from the job analysis done on a classification and the related content validity study for identifying the instruments to be used in the subsequent selection process.
These two concepts are intertwined in that the weight a selection instrument is given can impact the sequence of testing or the “Successive Hurdles” in the selection process. While the weighting process focuses on mathematical computations usually spelled out in the specific job analysis and test development system being utilized, sequencing of testing involves more administrative concerns.
Two considerations that should be utilized in establishing the order tests will be given are whether or not a test is considered pass/fail and the labor intensity of administering a particular test. The criticality of a particular knowledge, skill, ability, or personal trait will determine whether or not the test utilized to measure that particular KSAP, will be pass or fail. Pass/ fail as used in this sense indicates that success on the test demonstrates a sufficient amount of the required KSAP to will allow a candidate to continue in the selection process while a lack of success on the test will eliminate a candidate from the process. In that regard, as used here, pass/fail does not relate to whether or not a selection instrument is used for ranking, merely whether or not a threshold or pass point has been established that determines whether or not a candidate is permitted to continue to the next step. The usefulness of an instrument for ranking is determined by the validity of that instrument and whether or not it is capable of distinguishing levels of job performance and therefore supports ranking better test performers higher than those who perform poorer on the test.
Narrowly defined, “Successive Hurdles” can refer to the fact that any individual that is not successful in one portion of the selection process cannot proceed to the next portion. Since tests that compromise a test battery or selection process should all measure different KSAP’s and ideally correlate with job performance while not correlating with each other, all instruments in the battery should be used to determine whether or not a candidate is permitted to proceed to the next step. However; many systems, including those for police officer may necessitate that candidates continue to the next phase while aspects of previous testing are evaluated. This being the case, most jurisdictions functionally define “Successive Hurdles” as the process involved in leaping over each step in the process successfully whether or not success on previous instruments has been fully determined or not.
Combining the criticality of an instrument along with its ease of administration often results in a clear cut sequence of testing. Ideally, instruments that are relatively easy to administer in that they are not particularly labor intensive will also measure critical aspects of job performance or basic job requirements and therefore make clear choices for being administered first in the process. Perhaps the best example of this combining of the two concepts is found in the instruments designed to evaluate a candidate’s background. While conducting a thorough background investigation is a time consuming and labor intensive process, a quick inventory or “mini” background check that focuses only on immediate and legally defensible disqualifiers can be administered easily at the beginning of the selection process to narrow the applicant pool and reduce the costs of administering the next steps in the process. For example, in order to meet the basic Peace Officers Standards and Training in Nevada, which essentially means an individual can be admitted to a training academy for police officer, the individual must be twenty-one years old, a citizen of the United States without any felony convictions or convictions for domestic battery. This means that an instrument that determines eligibility to enter a training academy could be administered rather inexpensively while having a significant impact on the applicant pool.
Inserting the next steps in the process would involve evaluation of the same two concepts. Typically, a logical second step is a written exam in that it can be administered to a large number of applicants with the use of a relatively few proctors. Other instruments such as physical fitness tests, oral boards, psychological evaluations, polygraphs, background investigations, and medical exams require an almost one to one ratio between applicants and those administering the test and therefore are more labor intensive and more expensive. Therefore, it makes sense to administer these instruments in an order that reflects their costs per candidate while taking into consideration the number of candidates succeeding in or failing these tests. In that regard, it is also important to determine when a sufficient amount of information has been gained on a candidate to determine whether or not it is feasible to make a conditional offer of employment so that additional testing can be done without violating any provisions of the Americans with Disabilities Act (ADA).
The use of multiple selection instruments that necessitate the use of the “Successive Hurdles” model is a sound selection process that is often necessitated by the complexity of the KSAP’s that must be measured. Using this model wisely and effectively is a valuable administrative tool that can save agencies money while assisting in selecting the best candidates available.
This is part one of a four-part series on successive hurdles, test weighting and certification rules. Part 2 will focus on weighting selection instruments within the selection process and will be available to read on the Assessment Services Review on April 11, 2012. Part 3 will be available on April 18, 2012. The series will conclude with part 4 on April 25, 2012. In case you missed it, check out Robert Burd’s previous series, Item Analysis In Public Safety.
