What Are The Characteristics of a Good Test?

Part 1 in the Validity of Public Safety Assessments Series

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment.” As my forgiving readership already knows, I have trouble containing my thoughts to a single entry. So, as I began to frame out how I would respond to the question of the validity of public safety assessments, the amount of material I wanted to cover started to grow exponentially. At some point, I decided it would be best to start from the beginning with a series of primers on topics related to validity, building up to an answer to the question of “what is the validity of public safety assessments.”

So now this blog will be the first in a series looking at this question. Over a series of articles aimed to inform, but also intended to keep things simple, I will cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Are public safety assessments good tests and are they valid?

This first article in the primer series deals with the question of what is a good test. A good test can be defined as one that is:

  • Reliable
  • Valid
  • Practical
  • Socially Sensitive
  • Candidate Friendly.

Briefly and simply, I will review the meaning of each of these characteristics. Continue reading

Entry-Level Firefighter Test (Discounts Available!)

Has your agency previously administered any of the tests from IPMA-HR’s Entry-Level Firefighter (FF-EL) series?

If you answered ‘yes’ to this question, then we need your help!

IPMA-HR is gathering the test scores received by current firefighters who took the FF-EL test to better understand how the tests are currently performing.

What We’ll Need from You:

  • Firefighter’s FF-EL 100-, 200-, and/or 300-series test scores between 1994 and 2016.
  • Fire Academy score (if applicable)
  • Completion of a 9-question web-based performance evaluation

What You’ll Receive from Us:

  • $100 off your agency’s next test order
  • Direct evidence supporting the validity of the exam and its use in your agency
  • Hire better. High quality tests help you select high-quality candidates, saving your agency the costly expense of turnover.
  • And it’s free! A test development and validation project of this scale done internally could cost your agency in excess of $100,000. Participation in our test development projects is free.

Please contact Julia Hind-Smith at jsmith@ipma-hr.org if you would like to learn more and/or participate in this project.

(Please Note: All information gathered for this study will remain strictly confidential.  The data gathered will be combined with data from other departments and only be reported in the form of group statistics.)

A Roundup from Across the Web

times

This article talks about how the Victoria Fire Department in B.C. just got permission to use drones during emergencies as a first response tool. They will be using the drones during earthquakes, fires, and search and rescue missions. By having the drones, they can make sure that any building they are going into will be secure enough to withstand firefighters inside and check to see if there is anyone stuck inside of a burning building without risking firefighter lives. Search and rescue missions have now changed with the advantage of having drones. For example, if a child has gone missing in the water, the drone can hover over and find the exact location of the child without wasting time or funds to get a helicopter up and running. In all, drones seem to provide a strategical advantage to the future of fire and rescue missions.  Read more… Continue reading

Legal Update: including Physical Abilities Testing and Big Data

Physical Abilities Testing

There were recent developments in two cases related to the use of physical abilities tests as part of the pre-employment process.

United States of America v. Commonwealth of Pennsylvania et al.

In August 2016, the U.S. Department of Justice (DOJ) asked a federal judge in the U.S. District Court for the Middle District of Pennsylvania to grant a partial summary judgment indicating that the physical abilities test battery used by the Pennsylvania State Police (PSP) disproportionately screened out female applicants clearing the way for trail against PSP. The DOJ claims the court should conclude that the state’s use of its physical abilities test for screening entry-level state police trooper positions has a disparate impact on women and should therefore allow the parties to proceed to trial to determine whether the test is job-related and consistent with business necessity or whether there is a less discriminatory alternative employment practice that serves the state’s needs. Continue reading

Out With The Old, In With The New Year (2017 Edition)

Traditionally, I have started the New Year with a blog that recaps the past and looks to the future in assessment.  This year we say good bye to 2016, and enter 2017.  Of course, the big news in the United States was the election of a new President.  I am not bold enough to claim I can predict how a new administration will impact public sector human resources.  However, I do believe that I can make a prediction regarding the three hot trends for next year, and, they are each a carryover from the past several years.

My habit has been to insert a statement concerning how difficult it is to predict the future. However, this year I was surprised to find that many of the topics I would select for future trends, were actually covered in my blogs over the past year. So, maybe I am getting better at prophecy with advancing age.

My predictions for future trends or hot topics over the coming year include:

  • Big Data and Predictive Analytics.
  • Emerging Technologies.
  • Police Performance.

Continue reading

Improving the Interview, Part 2b: Managing the Panel Interview

In this blog, I will respond to what I see as practical questions that often arise in planning for a panel interview. I do apologize for the delay in the production of this third, and final, blog on the interview. Unfortunately, at times, real life intervenes.

I started this series by noting that no other selection device is as ubiquitous as the interview, while at the same time as misunderstood. Then, in Part 1, I discussed the individual selection interview. In Part 2, I discussed panel interview, including the availability from IPMA-HR of a product known as the Police Officer Structured Interview System or POSIS.

This month, in the third and final blog, I respond to what I see as some frequently encountered questions regarding the panel interview including:

  • Should I train raters?
  • Who should be on the panel?
  • How should I combine ratings to arrive at a final score?
  • What type of records should I keep?
  • How long should it take?

As a warning, a lot of my answers will involve a combination of “it depends” and “on your local rules or procedures.”

The Panel Interview

Continue reading

New Test Versions and Reading Lists Available (PSUP, PL, PDET, FCO)

Due to recent updates to the published source material for our series of Police Supervisor (PSUP), Police Lieutenant (PL), Fire Company Officer (FCO), and Police Detective (PDET), we have decided to update several of the test questions that appear in those test forms.  In addition, we have updated the reading lists belonging to each test to reflect the most current published source material.

To assist our customers in more easily recognizing which test version they are administering, we have reprinted the new tests under new names. The new tests correspond to the old ones and have undergone changes as follows:

If you’ve already distributed the reading lists for any of the older versions of these tests, then you do not need to take any further action to redistribute a new reading list. When you order your test, just make sure to order the test that matches the reading list you provided to your candidates. We will retain sufficient stock of the existing tests to accommodate those processes that have already begun.

At this point forward, we will be providing the updated reading lists for all new orders. If you’d like to request the new reading list, you may do so here: http://ipma-hr.org/assessment/support/readinglist.

Please contact us if you have any questions or concerns about this update.

For more information on each of these tests, please visit: http://ipma-hr.org/assessment/tests/police

Big Help Deserves Big Rewards

rewards

Help IPMA-HR with the criterion-related validation phase of our new public safety telecommunicator (PST) test, and you’ll not only be helping to ensure the continued availability of the most reliable, valid and fair tests in the industry, but you and your department will also receive the following rewards:

  • 30% off your agency’s next test order.
  • $75 VISA gift card for each participant.
  • A snack basket for participants to share after the test.
  • $50 Applebee’s gift card for the test administrator.
  • Direct evidence supporting the validity of the exam and its use in your agency.

Learn more about the new test, the validation phase and how you can help on our website.

Providing you with tests you can trust is our priority. But we can’t do it without you. Go online today to learn more and fill out a participation form for your agency.

Please Note: Our desired deadline for wrapping up the validation phase of this study is the end of October 2016, so please get in touch soon!

Improving the Interview Part 2a: The Panel or Board Interview

In the last blog, we investigated possible improvements that could be made in the use of individual interviews in pre-employment or promotional screening.  This month we expand our discussion to include the panel or board interview, an approach used by many public sector organizations.

As is often the case, once I start on a topic I have trouble controlling myself and my word count quickly gets out of control (my students have learned that if you ask me a simple question it can easily turn into an hour-long response). So, I have divided this blog into a 2a and 2b. In 2a, which you are reading right now, I:

  • Delineate the major characteristics of the panel interview;
  • Offer a version of a panel interview checklist;
  • Discuss the need for structure and training;
  • Provide an overview of the IPMA-HR Police Structured Interview System (POSIS).

Then, in a soon-to-follow Part 2b, I will answer frequently asked or encountered questions regarding the panel interview. Continue reading

We need writers!

writing

Are you well-versed in the world of assessment? Need an outlet through which to share your plethora of knowledge? Seeking opportunities to flex your writing muscle? Want your own byline? Look no further! IPMA-HR is looking for paid contributors to the Assessment Services Review: the official blog of the Assessment Services Department. We want to offer our readers fresh perspectives, experienced commentary, and cutting edge news from our world. Your level of commitment is up to you: write one, two … or 12 entries a year — and get paid to do it!

Interested? Please send an email to Andrey Pankov at apankov@ipma-hr.org