The City of Chicago ran an entry-level firefighter test in 1995. After many subsequent uses of the results, a court ruled in 2004 that the test was discriminatory, based on race. The city did not try to defend the test. Instead, city officials argued that discrimination charges were time-barred. Finally in 2010 the U.S. Supreme Court agreed in Lewis v. Chicago that the test was bad but found the charges were good. The Chicago Tribune reported on August 17, 2011 that the decision will cost the city $45M. This does not include the city‘s legal costs.

In Ricci v. Destefano the City of New Haven not only did not defend its firefighter promotional tests, but portrayed them as somehow flawed to avoid certifying promotional lists. It was not until the final Supreme Court substantive brief that the city asserted that there was a problem with the tests’ validity. The Court ordered the city to live with the results. Along the way some well known industrial-organizational psychologists filed an amicus brief asserting that the tests could not be valid because they were content-deficient. And now the Second Circuit in Briscoe v. New Haven has cleared the way for the discrimination suit the city said it was trying to avoid. The court noted that Briscoe is arguing, among other things, that the “industry norm” for firefighter promotion is 30% for the written test and 70% for the oral component, rather than the 60/40 weighting that the city used.

New York City’s entry-level firefighter test went down without much of a fight in United States. v. New York, with the judge blasting the city for not following the well-established precedent from 1980, Guardians v. Civil Service NYC. The judge granted summary judgment because no reasonable trier of fact could find for the city. In Connecticut the state’s physical ability test for Correction Officer was about to be tossed out. From what the judge said in Easterling v. Connecticut Department of Correction, the state did not seem to have a good idea why it was using a timed run as part of the test and what the cut score should be. And then came news that New Jersey was going to settle rather than fight regarding challenges in United States v. New Jersey Civil Service to its police sergeant’s exam, used by several municipalities in the state.

The allegations raised by the Department of Justice’s experts in the New Jersey case are a recital that could have come from 30 years ago. There is statistically significant adverse impact. The subject matter experts for the job analysis were not representative across jurisdictions using the test.

The multiplicity of tasks and worker elements rendered the ratings and linkage unreliable. There was no clear test plan. Anything that did not reduce to a knowledge element was ignored. There was no basis for the cut score on the written test. During 2000-2009, the cut score was always reported as 70, but it was transformed to bring about compliance with the Four Fifths Rule. (Any employer who thinks that the Four Fifths Rule provides a safe cut score, especially in the Third Circuit—home of Lanning v. SEPTA—is definitely living in the past.)

The state‘s expert, with as much experience in the testing business as the plaintiff‘s, offered that the adverse impact was not as bad as it might have been. But it seemed that there remained issues for which the state did not have a ready answer.

I learned my craft with New Jersey Civil Service, and there were many good professionals to learn from. Back then we were confident that we could clean up the tests and eliminate adverse impact. We‘re not that naïve now. But how is it that the issues known then are seemingly ignored currently? Whatever the reason, the negative consequences should be obvious. Applicants think they have been deprived of their rights and sue for redress. Drawn-out litigation over promotions can lead to unfilled positions and lower morale. Judges get a chance to meld their understanding of testing professional principles with their understanding of the law, sometimes with weird results that will impact new situations. The resolution of the litigation may result in a selection system less than optimal for staffing in the interest of the public good. The jurisdictions—which is to say, the taxpayers—have badly needed money diverted from public services to pay for legal expenses, settlements, and damages suffered by the successful plaintiffs.

For the problem to persist for so long indicates that it is bigger than any single jurisdictions. Three decades ago there was recognition of the magnitude of the problem with federal help available through the Intergovernmental Personnel Act. That aid is gone. Some of the consortia that were encouraged as part of that effort have withered away. I have no claim to the wisdom to resolve the matter. But I do think that the wisdom is out there, in the associations of individual professionals and the consortia of testing jurisdictions.

Instead of just the articles and presentations of good practice from individual jurisdictions, what if there were the consensus conclusions on good practice specific to public safety from a working group of testing experts, personnelists, and attorneys that was presented to, and fine-tuned and ultimately adopted by, the professional groups? What if the standards that emerged produced a brighter line on what practices would be dead on arrival if they went to court, and what arguments produced by “hired guns” for either side would be tossed out by judges as not reflecting generally accepted professional practice? What if the elected officials of the jurisdictions were provided with clearer statements of the issues in selection, and the judiciary provided with clearer guidance technical matters and expert testimony that could arise in court cases?

I don‘t have illusions of the difficulties in doing this. This discussion has only mentioned professional considerations, not political and financial context. But the outcome could not be worse than the lack of progress that seems to be reflected in the status quo, a reflection of where we were 30 years ago.

 

Court Cases Referenced:

Briscoe v. City of New Haven, No. 10-1975-cv (2nd Cir. 8/15/2011).
Easterling v. State of Connecticut Department of Corrections, No. 3:08-cv-00826 (D. Conn., 5/6/2011).
Guardians v. Civil Service Commission of the City of New York, 630 F.2d 79 (2nd Cir. 1980).
Lanning v. SEPTA, 181 F.3d 478 (3d Cir. 1999).
Lewis v. City of Chicago, No. 08–974 (S. Ct. 5/24/2010).
Ricci v. Destefano, 129 S. Ct. 2658 (2009).
United States v. City of New York, No. 07-CV-2067 (E.D. N.Y. 1/13/2010).
United States v. State of New Jersey, No. 2:10-cv-00091 (D. N.J. 2011)

Reprinted with permission from the Personnel Testing Council of Metropolitan Washington.