At last, we have reached the end of the employment law series. It is important to note that the four cases discussed in this series are in no way exhaustive of all the legal cases that exist with implications for test development and usage. They are simply the major and most often referenced cases.

As test users, if you ever find yourself asking “Am I allowed to do this” before, during or after a test administration, chances are there are legal cases and/or professional guidelines that can answer your question. If you’re still feeling uneasy, try asking the test developer or your legal team for advice. The most common questions I personally receive are:

Although this exam was designed for X position, could I also use it for a similar Y position?

Where should I set my cut score?

How should I determine who passes the exam?

Test validation can be a confusing topic, and it is my hope that this series of posts has clarified the purpose and importance of test validation for test users and developers. I like to think of test validation as the process of gathering evidence that your test is job-related. What does “job-related” mean and how do you gather this evidence? This is where the court cases and legal/professional guidelines really come in handy. For example, the case of Washington v. Davis (1976) involved debate over the strategy used to provide support for job-relatedness — reviewing the Court’s decision on this case can help provide insight into what is likely to be accepted in Court. Other useful resources will be listed at the end of this post, but for now, let’s get started with Ricci v. Destefano (2009).

Ricci v. Destefano, 557 U.S. 557 (2009)

New Haven, Connecticut’s Fire Department required all firefighters seeking promotion to captain or lieutenant to pass a written examination that was developed and validated by a test development company. The African American and Hispanic/Latino firefighters received significantly lower scores on the exams than the White candidates for the position, which would have resulted in none of the minority test takers being eligible for promotion based on their test scores. The Black and Hispanic/Latino applicants then threatened to sue the city of New Haven if they used the results from the test to make promotional decisions and demanded that the city invalidate the test on the basis of adverse impact. Fearing litigation, the city of New Haven complied and settled the case without litigation. The test results were ignored, and no candidates were promoted based on the results.

The White firefighters who would have received promotions based on their passing score were not happy with the city’s decision to invalidate the results and argued that this was unfair due to the exam having been properly validated and administered. They brought their case to the U.S. Supreme Court for verdict.

The Decision: The U.S. Supreme Court ruled that although the results of the exam would have resulted in disparate impact against Black and Latino/Hispanic firefighters, it was unlawful for the city of New Haven to invalidate the results due to the exam having been properly validated and administered.

The Implications: Needless to say, this court decision in particular was controversial and perhaps confusing, but let’s try to clear some things up. First, the Court made it clear that adverse impact alone is not an indication of an invalid test or that a test should not be used for hiring or promotional purposes. This is important because “adverse” or “disparate impact” are words that cause many test users to shudder due to the negative connotation. To avoid using the results of an examination that has sufficient evidence to support it’s use due to fear of adverse impact litigation is therefore unwise for test users, and additional steps need to be taken.

The Court also made clear that when adverse impact is found, the burden of proof shifts to the test user (the city of New Haven) to defend the use of the test as a “business necessity.” That is, the city of New Haven needed to prove that they needed to use the test to be able to properly determine who is qualified and eligible for promotion. Once business necessity has been proven, the city then needs to prove that they investigated alternate means to make the promotional decision that could have achieved the same result without adverse impact. If no alternate means (or alternate test) were found that could achieve the same result without adverse impact, then the Court is likely to rule in favor of the test user.

Test developers and test users should attempt to conduct adverse impact analyses when data permits. Test developers should make this information available to test users, and test users would also benefit from submitting their candidate data to the test development company. These analyses will help to first determine if adverse impact has historically been indicated. If your agency will be using an examination that is known to have adverse impact, be sure to maintain the evidence you need to prove that using the exam is a business necessity (job analysis information, technical reports, etc.). Test users are also encouraged to investigate many alternate testing means before making their final decision on which test to use — doing so also provides support for using your chosen exam.

References

Ricci v. DeStefano, 557 U.S. 557 (2009)

Other useful resources

SIOP’s Principles for the Validation and Use of Selection Procedures (2019)

APA, AERA, and NCME’s Standards for Educational and Psychological Testing (2014)

Uniform Guidelines on Employee Selection Procedures (1978)

Adverse Impact: Implications for Organizational Staffing and High Stakes Selection (2010) (Edited by James L. Outtz)