Joint Agency Police Promotional Assessment Centers: An Opportunity for Interagency Cooperation

The current fiscal environment for many law enforcement agencies requires innovative change in practices and procedures.  While we may not be facing the stark challenges of retrenchment budgeting that were common in the 1980s, today many of us have insufficient budgets to achieve all that we seek to accomplish. A joint agency promotional assessment center may enable multiple agencies to realize the benefits of the assessment center at a substantial cost savings.

Background Brief

Human Resource Management costs are significant in public policing.  As a rule of thumb 70-90% of any law enforcement agency’s budget is committed to some aspect of Human Resource Management (HRM).  These costs have been predicted to grow in the future as a result of any number of factors.  But a common factor in any HRM cost equation is the identification, selection and retention of competent and capable supervisors and managers.

Across the nation, several law enforcement agencies have adopted the promotional assessment center as a defensible and fair tool in the selection of supervisors and managers.  However, as many readers know only too well – a functional promotional assessment is an expensive tool[i].   The costs present in any one of several forms – including the significant time commitment of both the candidates and assessors; the logistical support required for a successful center (video recording, appropriate exercise spaces, and scheduling trained actors) as well as the extensive and intensive planning required for a successful assessment center process. Continue reading

Participants Needed for Entry-Level Firefighter Test Study (Discounts Available!)

IPMA-HR is in the final stretch of finishing up the validation study on our entry-level firefighter (FF-EL) test series, but we still need your help!  We are seeking a few more agencies who have administered a test from the FF-EL series to provide us the scores received by firefighters who were selected for the job.

Once we’ve received these test scores from you, we ask that each firefighter’s supervisor complete a brief 9-question performance evaluation.  This will allow us to see whether our tests from the FF-EL series continue to be successful in predicting on-the-job performance.

For your help, you will receive $100 off your agency’s next IPMA-HR assessment order.  In addition, your participation in this project will provide direct evidence supporting the validity of these exams and their use in your agency.

If you would like to participate, please visit the following link to provide your test scores:

https://www.surveymonkey.com/r/FFELScores

Our Research Associate, Julia Hind-Smith, will then follow up with you to provide further instructions on completing the performance evaluations.  If you have any questions about the study, please feel free to reach out to Julia at jsmith@ipma-hr.org.

(Please Note: All information gathered for this study will remain strictly confidential.  The data gathered will be combined with data from other departments and only be reported in the form of group statistics.)

Should We Be Gamifying Our Assessments?

For this month’s blog, I will look at what is one of the hottest questions in private and public-sector personnel selection – “Should we be gamifying our assessments?” In my opinion, the answer is “Yes!” and I will take this blog to explain why.

“Gamification” falls within the larger topic of “how should I incorporate emerging technologies into my assessment strategies.” Now, one might legitimately ask how it is that someone who started doing math on a slide rule can claim to be an expert on emerging technologies. I will simply remind you that Steve Jobs, Steve Wozniak, Paul Allen, Bill Gates, and I were born at roughly the same time. So, despite huge differences in our net worth, we do share a similar generational zeitgeist.

What Does Gamifying Mean?

Gamifying is one type of Technology Enhanced Assessment (TEA). Related types of TEA include:

  • games (so obviously there is a difference between games and gamifying).
  • enhanced item types.
  • the use of avatars.
  • virtual reality.
  • big data and advanced algorithms.

Gamification (or gamifying) is defined as “the application of game mechanics, elements, and features to non-game environments,” or in this case “the application of game-type elements to assessments used in personnel selection.” This differs from the use of true games in selection, although the difference is probably more of a continuum than a sharp distinction, as both games and gamification can be used in personnel selection. The differences between games and gamification can be summarized as:

  • With games, the person knows they are playing a game, whereas with gamification, the applicant still knows they are taking a test.
  • Games are meant to be fun and are structured to have clear rules that define the game play; gamified tests are seen as a more serious activity.
  • Games have different play sections and winning is the goal; gamified tests are structured similar to traditional assessments and getting hired is the goal.

Continue reading

Participants Needed for Office of Fire Marshal Study

IPMA-HR is currently seeking participants for a nationwide Office of Fire Marshal/Office of Fire Prevention study. This study is the first step in the development of a new Fire Marshal Test. The first part of the study involves surveying current members of the Office of Fire Prevention to learn about the important duties and demands of their job.

Examples of applicable positions include, but are not limited to: Fire Marshal, Fire/Arson Investigator, Premise Officer, Deputy Fire Marshal, Fire Prevention Officer, Fire Inspector, and Code Enforcement Officer.

If you or someone you know holds one of the above positions or a similar position, we would greatly value your input in developing our newest test series.

Participants will be entered into a raffle with a $500 prize! Participating agencies will also receive a 15% discount toward a future IPMA-HR assessment product purchase. Interested parties can use the following link to participate:

 https://www.surveymonkey.com/r/FireMarshal

Questions can be emailed to our Research Associate, Julia Hind-Smith, at jsmith@ipma-hr.org.

Are Tests Valid for Public Safety Jobs?

Number 4 in the Validity of Public Safety Assessments Series

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment?” In response, I decided to do a series of articles aimed to inform, but also designed to keep things simple. The blogs in this series were intended to cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Finally, trying to address the original question I was asked, are public safety assessments valid?

The first three blogs in the primer series have been published and are available by clicking the links above.

This is the fourth and final article in the series and is intended to answer the question regarding the validity of tests for public safety jobs. I define public safety jobs here as including police, fire, and emergency medical services (EMS). In addition, human resource professionals are usually interested in the use of tests in both entry level screening and for arriving at promotion decisions. Continue reading

What is Validity?

Number 3 in the Validity of Public Safety Assessments Series

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment?” In response, I decided to do a series of articles aimed to inform, but also designed to keep things simple. The blogs in this series are intended to cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Are public safety assessments good tests and are they valid?

The first two blogs in the primer series has been published and are available by clicking the links above.

This is the third in the series and is intended to provide a basic introduction into the various kinds of validity evidence. By validity evidence, I do not mean the obvious distinction between the big four of:

  1. Content
  2. Criterion-related
  3. Construct
  4. Transfer or Transportability

Understanding the distinctions between the four types of validity listed above is important. However, in this blog, I mean something different by types of validity evidence. As our ultimate purpose or goal is to respond to the question as to whether tests are valid for purposes of public sector assessment, we can consider the following five types of validity evidence as relevant:

  1. Local Validation Based on Criterion-Related Evidence
  2. Validity Generalization Evidence Based on Tests in General
  3. Validity Generalization Evidence Based on Specific Occupation
  4. Validity Generalization Evidence Based on Specific Test
  5. Other

Continue reading

Where Can I Find Guidelines for Tests?

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment?” In response, I decided to do a series of articles aimed to inform, but also intended to keep things simple. The blogs in this series are intended to cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Are public safety assessments good tests and are they valid?

The first article in the primer series has been published and is available on the IPMA-HR Assessment Services Review page.

This is the second in the series and is intended to answer the question as to where can the reader turn for guidance in addition to that offered in this series of blogs. My suggested list is fairly short and includes:

Continue reading

Police Promotional Assessment Centers Selection: Then What?

In small and medium sized law enforcement agencies, opportunities for promotion are periodic at best.  Often years will pass before another promotional opportunity is available.  It is in our professional best interest as leaders to prepare as many qualified individuals as possible for the next promotional opportunity.

In policing, a promotion is not only a personal and professional accomplishment, but a very public marker of a heathy organization.  Promotions are indicative of change and with change can come growth.  But there is risk in every promotional process.  We select those for promotion through any one of several promotion processes with the singular goal of identifying those best prepared to assume the role and responsibility of the next higher rank.

But a question that must be considered is – and then what? Routinely that individual selected for promotion begins an orientation and onboarding process for his/her new role in the organization.  But what about those who are labelled as the ‘not selected’?  What is to come for those men and women who are not promoted? Continue reading

What Are The Characteristics of a Good Test?

Part 1 in the Validity of Public Safety Assessments Series

The idea for this primer series germinated from a simple question – “Could you do an article looking at the validity of tests used in public safety assessment.” As my forgiving readership already knows, I have trouble containing my thoughts to a single entry. So, as I began to frame out how I would respond to the question of the validity of public safety assessments, the amount of material I wanted to cover started to grow exponentially. At some point, I decided it would be best to start from the beginning with a series of primers on topics related to validity, building up to an answer to the question of “what is the validity of public safety assessments.”

So now this blog will be the first in a series looking at this question. Over a series of articles aimed to inform, but also intended to keep things simple, I will cover:

  1. What are the characteristics of a good test?
  2. What are some authoritative references human resource and assessment professionals can rely upon in evaluating the worthiness of tests?
  3. What is validity?
  4. Are public safety assessments good tests and are they valid?

This first article in the primer series deals with the question of what is a good test. A good test can be defined as one that is:

  • Reliable
  • Valid
  • Practical
  • Socially Sensitive
  • Candidate Friendly.

Briefly and simply, I will review the meaning of each of these characteristics. Continue reading

Entry-Level Firefighter Test (Discounts Available!)

Has your agency previously administered any of the tests from IPMA-HR’s Entry-Level Firefighter (FF-EL) series?

If you answered ‘yes’ to this question, then we need your help!

IPMA-HR is gathering the test scores received by current firefighters who took the FF-EL test to better understand how the tests are currently performing.

What We’ll Need from You:

  • Firefighter’s FF-EL 100-, 200-, and/or 300-series test scores between 1994 and 2016.
  • Fire Academy score (if applicable)
  • Completion of a 9-question web-based performance evaluation

What You’ll Receive from Us:

  • $100 off your agency’s next test order
  • Direct evidence supporting the validity of the exam and its use in your agency
  • Hire better. High quality tests help you select high-quality candidates, saving your agency the costly expense of turnover.
  • And it’s free! A test development and validation project of this scale done internally could cost your agency in excess of $100,000. Participation in our test development projects is free.

Please contact Julia Hind-Smith at jsmith@ipma-hr.org if you would like to learn more and/or participate in this project.

(Please Note: All information gathered for this study will remain strictly confidential.  The data gathered will be combined with data from other departments and only be reported in the form of group statistics.)