About Dennis Doverspike

Dennis Doverspike, Ph.D., ABPP, is a Full Professor of Psychology at the University of Akron, Senior Fellow of the Institute for Life-Span Development and Gerontology, and Director of the Center for Organizational Research. He is certified as a specialist in Industrial-Organizational Psychology and in Organizational and Business Consulting by the American Board of Professional Psychology (ABPP) and is a licensed psychologist in the State of Ohio. Dr. Doverspike has over thirty years of experience working with consulting firms and with public and private sector organizations. Services provided include individual assessments, statistical analysis, development of large scale assessment systems, job evaluation and job analysis, and expert witness services. He is the author of 3 books and over 150 other professional publications. Dennis Doverspike received his Ph.D. in Psychology in 1983 from the University of Akron. His M.S. in Psychology is from the University of Wisconsin-Oshkosh and his B.S. is from John Carroll University.

Out With The Old, In With The New Year (2017 Edition)

Traditionally, I have started the New Year with a blog that recaps the past and looks to the future in assessment.  This year we say good bye to 2016, and enter 2017.  Of course, the big news in the United States was the election of a new President.  I am not bold enough to claim I can predict how a new administration will impact public sector human resources.  However, I do believe that I can make a prediction regarding the three hot trends for next year, and, they are each a carryover from the past several years.

My habit has been to insert a statement concerning how difficult it is to predict the future. However, this year I was surprised to find that many of the topics I would select for future trends, were actually covered in my blogs over the past year. So, maybe I am getting better at prophecy with advancing age.

My predictions for future trends or hot topics over the coming year include:

  • Big Data and Predictive Analytics.
  • Emerging Technologies.
  • Police Performance.

Continue reading

Improving the Interview, Part 2b: Managing the Panel Interview

In this blog, I will respond to what I see as practical questions that often arise in planning for a panel interview. I do apologize for the delay in the production of this third, and final, blog on the interview. Unfortunately, at times, real life intervenes.

I started this series by noting that no other selection device is as ubiquitous as the interview, while at the same time as misunderstood. Then, in Part 1, I discussed the individual selection interview. In Part 2, I discussed panel interview, including the availability from IPMA-HR of a product known as the Police Officer Structured Interview System or POSIS.

This month, in the third and final blog, I respond to what I see as some frequently encountered questions regarding the panel interview including:

  • Should I train raters?
  • Who should be on the panel?
  • How should I combine ratings to arrive at a final score?
  • What type of records should I keep?
  • How long should it take?

As a warning, a lot of my answers will involve a combination of “it depends” and “on your local rules or procedures.”

The Panel Interview

Continue reading

Improving the Interview Part 2a: The Panel or Board Interview

In the last blog, we investigated possible improvements that could be made in the use of individual interviews in pre-employment or promotional screening.  This month we expand our discussion to include the panel or board interview, an approach used by many public sector organizations.

As is often the case, once I start on a topic I have trouble controlling myself and my word count quickly gets out of control (my students have learned that if you ask me a simple question it can easily turn into an hour-long response). So, I have divided this blog into a 2a and 2b. In 2a, which you are reading right now, I:

  • Delineate the major characteristics of the panel interview;
  • Offer a version of a panel interview checklist;
  • Discuss the need for structure and training;
  • Provide an overview of the IPMA-HR Police Structured Interview System (POSIS).

Then, in a soon-to-follow Part 2b, I will answer frequently asked or encountered questions regarding the panel interview. Continue reading

Improving the Interview Part 1: The Individual Selection Interview

The Interview. No other selection device is as ubiquitous, while at the same time as misunderstood. Like an A-list celebrity, all you have to say is “the interview” and everyone can tell you stories, generate an opinion regarding love it or hate it, and tell you why it has received too much (or too little) notoriety, press, and attention.

In the next two blogs, I will look at the topic of “Improving the Interview.” This month, we will discuss the Individual selection interview, which is conducted in a one-on-one setting between an interviewer and an interviewee. In the next blog, we will investigate improving the board or panel interview.

If Everyone Uses It, What Could Be Wrong? 

Can a technique that every organization uses really be that bad? Well, the problem with the interview is that early studies found that the typical unstructured interview (referred to as “unstructured” because the interviewer was left to conduct and rate the interview as he or she wished) was not very reliable or valid. That is, despite the beliefs of human resource personnel and supervisors, the traditional interview was not a very good indicator of talent, merit, or the best candidate for the job.

The saving grace for the interview was the finding that introducing structure greatly increased the reliability and the validity of the interview. Depending upon the particular study, adding structure to an interview could double its validity as a predictor of job performance, turning it into one of the more valid selection devices.

Structure of Questions and Rating Scales

Structure can be introduced both into the questions asked as well as the way in which interviewee performance is evaluated. In terms of the questions themselves, each candidate should be asked the same questions in the same manner. The questions should present the interviewee with a situation and ask how he or she would respond, or a candidate may be asked to describe how they may have handled a problem situation in a past job. Continue reading

Should I Provide Assessment Feedback?

The topic of my blog for this month deals with employers providing developmental feedback to candidates based upon the results of employment test or assessment.  Although the feedback of results from employment tests is common in many other countries, it is less frequently the case that such feedback is provided in the United States.

My topic this month deals with using assessment or test results in order to provide developmental feedback and suggestions to employees.  Although I will be dealing with feedback from tests in general, I will pay special attention to assessments that allow for a more in-depth, comprehensive view of the individuals, such as offered by the use of assessment centers.

[For more information on assessment centers, see Public Safety Assessment Center System (PSACS) and Assessment Center Educational Materials (ACEM)]

Some Findings from a Quick Literature Search

I had a graduate student perform a quick search of the current literature. Our findings regarding policies toward providing developmental feedback by employers in the United States were that it is rare for organizations to provide scores or give feedback to job applicants for pre-employment tests.  It is more common for promotional candidates, but even there the exact type of feedback may skew toward simply providing results or scores.  Providing expansive or detailed feedback is most likely to occur where the tests are used specifically for training or developmental purposes.

As for assessments centers, The International Congress on Assessment Center Methods has a document entitled The 2014 Guidelines and Ethical Considerations for Assessment Center Operations (6th Edition).  According to their guidelines, feedback should be provided and if the assesses are members of the organization than the employee has the right to “read any formal, summary, written reports concerning their own performance and recommendations that are prepared and made available to management.” Continue reading

Test Score Posting Policies in the Public Sector

My blog this month deals with what I believe is a complex question that requires deft consideration of the demands of multiple stakeholders and the careful weighing of legal and ethical issues. I am speaking of the question of how does a public sector jurisdiction make decisions regarding the posting of scores both during and after the completion of an assessment or selection project.

As human resource and assessment professionals, we have to resolve the conflict between the equally important values of transparency of feedback to test takers, the privacy rights and expectations of confidentiality held by job or promotional candidates, and the public’s right to know, along with the media’s right to information. Deciding how and what type of information to post can seem like a judgment worthy of Solomon, as the human resource professional must reconcile:

  • the public’s right to information, including possible public record laws;
  • the candidate’s desire for feedback and test score information; and
  • the right of the candidate to privacy and the candidate’s expectation that their scores will be handled in a confidential and sensitive manner.

In my opinion, one of the complicating factors is that the release and posting of test score information has to consider many factors beyond simple psychometric and assessment issues. Some of the factors that must be considered or questions which need to be asked and answered include:

  • Are there federal or state laws that govern the release of public sector employment test information, as well as public records in general?
  • Are there local Civil Service regulations or rules?
  • Does the union contract specify how test results will be issued?
  • Are there past, relevant court decisions?
  • How have we done it in the past? What are the existing precedents?
  • What precedent, if any, do we want to create for future tests?

My own experience has been that every jurisdiction tends to make decisions regarding the posting of and release of a candidates test and score information differently, even within a specific geographic area such as Northeast Ohio. I know of some cities that post in public all the test score information for each candidate, while similar nearby cities post only the final rankings of the test takers.

If at this point you are starting to mumble to yourself, “I fear that Doverspike is not going to give us a simple answer in this blog,” you are correct. However, I am going to share with you some data from a survey conducted by IPMA-HR Assessment Services. Continue reading

Job Analysis – What’s New?

What’s new in job analysis? A cynic might reply – “very little.”  However, such a conclusion would lead to a very short blog and, more importantly, would not be accurate.  Despite the foundational nature of job analysis, there have been some recent developments worth sharing.

Consensus on Recommended Practices

Although it is still true that the Uniform Guidelines and courts show no preference for any specific method of job analysis, due to pressures for documentation from regulatory agencies, a professional consensus has begun to evolve and emerge around recommended practices for job analysis.  The associated principles can be expressed as follows.  A job analysis should:

  • Be task-based. Despite continued mention of worker-oriented approaches, including the emergence of competency models, the job description should be task-oriented including detailed listings of tasks and associated knowledge, skills, abilities, and personal characteristics (KSAPs).
  • Identify linkages. The identification and measurement of linkages between tasks and KSAPs is critical.  When job analysis is used in test development, it is equally important to establish linkages between the KSAPs and the test content.
  • Utilize interviews and focus groups. The appropriate use of interviews or focus groups remains important in obtaining job information from incumbents and supervisors.
  • Incorporate questionnaires. Where practical, with practicality a function primarily of the number of incumbents and the quality of the information obtained from the interviews, questionnaires should be used to gather quantitative ratings of tasks, KSAPs, and linkages.  The collected data can then be subject to statistical analysis.  Technological developments, including the widespread availability of easy-to-use online survey software, have made it much simpler and cost-effective to create and distribute job analysis instruments.  In designing surveys, practitioners should be aware of the now ubiquitous nature of smartphones.  Large matrices of the type so frequently used to collect job ratings do not translate well to small screens.  As a result, analysts must be creative in designing surveys when the incumbents will be responding using mobile devices, including tablets and smartphones.

Continue reading

To Retest or Not to Retest: Answers (Part 2)

In our previous blog, I reviewed the research literature related to the retesting of applicants. Summarizing our findings from Part 1:

  1. If someone takes a test again, his/her score will increase.
  2. If a group of individuals are retested, the rank-order will change.
  3. At least two months, but more realistically 6 months to a year, should be required between most retests.
  4. Given a candidate is willing, there seems to be no reason to limit retests. The issues are really whether to even allow a first retest and the time between retests.
  5. Under typical situations, where only a portion of the applicants may be taking the test a second time, the first administration will probably be the most valid; but there are many factors that may influence this conclusion. And, from 1 above, we would expect those taking the test a second time to have higher scores than the first time examinees.

This month, our goal is to arrive at some practical suggestions regarding practice based on professional and government guidelines, the public sector testing model, and the previously mentioned research findings in order to come up with recommendations for applied practice.  This will include a discussion of how we should determine a score for someone who is retested.  Continue reading

To Retest or Not to Retest; That is the Question (Part 1)

Well, not the only question. In this blog, we will consider a series of questions including:

  1. If someone takes the test again, will there score change?
  2. If a group of individuals are retested, will the rank-order of the scores change?
  3. How much time should there be between retests?
  4. How many retests should be allowed?
  5. Which test scores are the most valid for predicting performance?

My answers will be based primarily on the research literature.  However, retesting is one of those topics where the importance of the question to practitioners has far outpaced the quantity and applicability of the published research literature.

Continue reading

Readability of Assessments in a Digital Age (Part 2): Practical Issues

  • We want test questions that are very detailed, highly complex, engage the test taker, reflect the job, and, oh yes, are at a 6th grade reading level.
  • Grade level = 5.9. Reading level from paragraphs from a 3rd grade reader from 1960, as calculated by Word.

This is part two of a blog dealing with the measurement of readability and the establishment of appropriate reading levels. For purposes of this blog, readability can be defined as the ability of material to be comprehended by its intended audience.

In Part 1, we investigated approaches to readability based on:

  1. The measurement of grammatical features or readability formulas.
  2. The linguistic perspective.
  3. Job analysis.

In Part 2, we turn our attention to more practical issues such as:

  • How are readability indices used by assessment professionals?
  • What adjustments can or should be made when evaluating multiple-choice tests?
  • How has the changing nature of jobs impacted readability?

Continue reading