Are Tests Valid for Public Safety Jobs?

Number 4 in the Validity of Public Safety Assessments Series

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment?” In response, I decided to do a series of articles aimed to inform, but also designed to keep things simple. The blogs in this series were intended to cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Finally, trying to address the original question I was asked, are public safety assessments valid?

The first three blogs in the primer series have been published and are available by clicking the links above.

This is the fourth and final article in the series and is intended to answer the question regarding the validity of tests for public safety jobs. I define public safety jobs here as including police, fire, and emergency medical services (EMS). In addition, human resource professionals are usually interested in the use of tests in both entry level screening and for arriving at promotion decisions. Continue reading

RFP for Entry-level Police Officer Selection Test Due Date Extended to September 15

The International Public Management Association for Human Resources (IPMA-HR) invites fixed-price proposals from qualified consulting firms to develop a multiple-choice entry-level police officer selection test for its stock testing service.

This includes the creation of an exam plan from the job analysis data collected by IPMA-HR, and the utilization of questions from IPMA-HR’s test item bank. The consulting firm chosen is expected to conduct a criterion-based validation study that includes having participating police departments administer the test to incumbents and gathering performance evaluation ratings to statistically show that the test predicts performance on the job. This also includes a test item analysis to select the final test questions, criterion validation statistics, and a fairness analysis.

The firm is expected to prepare validation study materials and assist in getting police departments to participate by making calls and sending correspondence. Further, the firm will prepare a technical report summarizing the steps taken during the test development and validation study project.

Proposals are due by September 15, 2017, 5:00 PM, ET.

Learn more about the full scope of the project and obtain contact information on our website. Continue reading

What is Validity?

Number 3 in the Validity of Public Safety Assessments Series

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment?” In response, I decided to do a series of articles aimed to inform, but also designed to keep things simple. The blogs in this series are intended to cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Are public safety assessments good tests and are they valid?

The first two blogs in the primer series has been published and are available by clicking the links above.

This is the third in the series and is intended to provide a basic introduction into the various kinds of validity evidence. By validity evidence, I do not mean the obvious distinction between the big four of:

  1. Content
  2. Criterion-related
  3. Construct
  4. Transfer or Transportability

Understanding the distinctions between the four types of validity listed above is important. However, in this blog, I mean something different by types of validity evidence. As our ultimate purpose or goal is to respond to the question as to whether tests are valid for purposes of public sector assessment, we can consider the following five types of validity evidence as relevant:

  1. Local Validation Based on Criterion-Related Evidence
  2. Validity Generalization Evidence Based on Tests in General
  3. Validity Generalization Evidence Based on Specific Occupation
  4. Validity Generalization Evidence Based on Specific Test
  5. Other

Continue reading

Request for Proposals for the Creation of an Entry-level Police Officer Selection Test

The International Public Management Association for Human Resources (IPMA-HR) invites fixed-price proposals from qualified consulting firms to develop a multiple-choice entry-level police officer selection test for its stock testing service.

This includes the creation of an exam plan from the job analysis data collected by IPMA-HR, and the utilization of questions from IPMA-HR’s test item bank. The consulting firm chosen is expected to conduct a criterion-based validation study that includes having participating police departments administer the test to incumbents and gathering performance evaluation ratings to statistically show that the test predicts performance on the job. This also includes a test item analysis to select the final test questions, criterion validation statistics, and a fairness analysis.

The firm is expected to prepare validation study materials and assist in getting police departments to participate by making calls and sending correspondence. Further, the firm will prepare a technical report summarizing the steps taken during the test development and validation study project.

Proposals are due by July 31, 2017, 5:00 PM, ET.

Learn more about the full scope of the project and obtain contact information on our website. Continue reading

Where Can I Find Guidelines for Tests?

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment?” In response, I decided to do a series of articles aimed to inform, but also intended to keep things simple. The blogs in this series are intended to cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Are public safety assessments good tests and are they valid?

The first article in the primer series has been published and is available on the IPMA-HR Assessment Services Review page.

This is the second in the series and is intended to answer the question as to where can the reader turn for guidance in addition to that offered in this series of blogs. My suggested list is fairly short and includes:

Continue reading

Police Promotional Assessment Centers Selection: Then What?

In small and medium sized law enforcement agencies, opportunities for promotion are periodic at best.  Often years will pass before another promotional opportunity is available.  It is in our professional best interest as leaders to prepare as many qualified individuals as possible for the next promotional opportunity.

In policing, a promotion is not only a personal and professional accomplishment, but a very public marker of a heathy organization.  Promotions are indicative of change and with change can come growth.  But there is risk in every promotional process.  We select those for promotion through any one of several promotion processes with the singular goal of identifying those best prepared to assume the role and responsibility of the next higher rank.

But a question that must be considered is – and then what? Routinely that individual selected for promotion begins an orientation and onboarding process for his/her new role in the organization.  But what about those who are labelled as the ‘not selected’?  What is to come for those men and women who are not promoted? Continue reading

What Are The Characteristics of a Good Test?

Part 1 in the Validity of Public Safety Assessments Series

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment.” As my forgiving readership already knows, I have trouble containing my thoughts to a single entry. So, as I began to frame out how I would respond to the question of the validity of public safety assessments, the amount of material I wanted to cover started to grow exponentially. At some point, I decided it would be best to start from the beginning with a series of primers on topics related to validity, building up to an answer to the question of “what is the validity of public safety assessments.”

So now this blog will be the first in a series looking at this question. Over a series of articles aimed to inform, but also intended to keep things simple, I will cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Are public safety assessments good tests and are they valid?

This first article in the primer series deals with the question of what is a good test. A good test can be defined as one that is:

  • Reliable
  • Valid
  • Practical
  • Socially Sensitive
  • Candidate Friendly.

Briefly and simply, I will review the meaning of each of these characteristics. Continue reading

Entry-Level Firefighter Test (Discounts Available!)

Has your agency previously administered any of the tests from IPMA-HR’s Entry-Level Firefighter (FF-EL) series?

If you answered ‘yes’ to this question, then we need your help!

IPMA-HR is gathering the test scores received by current firefighters who took the FF-EL test to better understand how the tests are currently performing.

What We’ll Need from You:

  • Firefighter’s FF-EL 100-, 200-, and/or 300-series test scores between 1994 and 2016.
  • Fire Academy score (if applicable)
  • Completion of a 9-question web-based performance evaluation

What You’ll Receive from Us:

  • $100 off your agency’s next test order
  • Direct evidence supporting the validity of the exam and its use in your agency
  • Hire better. High quality tests help you select high-quality candidates, saving your agency the costly expense of turnover.
  • And it’s free! A test development and validation project of this scale done internally could cost your agency in excess of $100,000. Participation in our test development projects is free.

Please contact Julia Hind-Smith at jsmith@ipma-hr.org if you would like to learn more and/or participate in this project.

(Please Note: All information gathered for this study will remain strictly confidential.  The data gathered will be combined with data from other departments and only be reported in the form of group statistics.)

A Roundup from Across the Web

times

This article talks about how the Victoria Fire Department in B.C. just got permission to use drones during emergencies as a first response tool. They will be using the drones during earthquakes, fires, and search and rescue missions. By having the drones, they can make sure that any building they are going into will be secure enough to withstand firefighters inside and check to see if there is anyone stuck inside of a burning building without risking firefighter lives. Search and rescue missions have now changed with the advantage of having drones. For example, if a child has gone missing in the water, the drone can hover over and find the exact location of the child without wasting time or funds to get a helicopter up and running. In all, drones seem to provide a strategical advantage to the future of fire and rescue missions.  Read more… Continue reading