Along with establishing tests under guidelines that ensure their validity and reliability, it is important to administer tests in a consistent manner across candidates to avoid diminishing their validity and reliability. In other words, the conditions under which a test is given should be standardized. Standardization of test administration can mean many things to different people and can vary from jurisdiction to jurisdiction, but the key is making sure that all candidates are treated as consistently as possible. In my experience, I have seen jurisdictions err in both directions, with some being too lax and some being too strict. The key is to avoid changes in the way a test is administered that could actually negatively impact candidates’ ability to perform at their optimum.
In that regard, it is also important to distinguish the concept of standardizing the manner in which a test is administered from what candidates often term, “fairness.” It is not uncommon for test takers to confuse what they consider to be “fairness” with what test administrators know to be standardization since test takers are unfamiliar with what actually constitutes standardization. While the two concepts are related, they are not the same. Fairness is a concept that exists in an individual’s mind and can often include obscure factors that have nothing to do with standardization. It is important for test administrators to understand factors related to standardization to avoid getting dragged into debates about the “fairness” of a test.
Quite often, written tests that are leased from a reputable test developer will provide relatively strict guidelines regarding how their tests are to be administered. These guidelines are valuable for establishing the standardization of test administration which can be used for leased tests as well as tests that are developed in-house. In addition, this type of standardization helps prevent challenges and complaints regarding test administration.
Typical standardization processes focus on the environment in which an exam is given and the opportunity given to all candidates to perform their best. Factors that are usually controlled include:
- lighting
- sound
- time
- room temperature
- instructions provided
- test booklets
- answer sheets
- pencils
- scratch paper
- use of calculators
- responses to questions
It is a good idea to compile lists of frequently asked question and develop standard responses to them. Commonly asked questions typically involve candidates looking for clarification to a question or asserting that there isn’t a correct response to a question. Agencies can avoid these questions by including statements in the overall instructions that indicate that the test has been reviewed and edited to ensure that each item has a correct response and proctors are not permitted to provide any additional information regarding individual items.
Candidates can also be told during the initial instructions what to do if they do not think there is a correct answer to a question. For example, tell them to write that information in the margin of the test next to the question and let the test administrator know before leaving so the question can be checked before scoring begins. Or, agencies can wait until specific questions are asked and then respond to them with the standard answers. If individual questions are permitted during a test session, proctors should be prepared to repeat the question and answer to the entire group so that everyone has the same information regarding the test.
Having a set plan for dealing with applicants who arrive late and having provisions that require applicants to provide proper identification both contribute to test standardization. Many jurisdictions provide a grace period of ten to fifteen minutes for candidates who have a legitimate reason for arriving late to a test session. However; a decision will have to be made as to whether candidates that do arrive late should be provided any additional time after the test session ends to make up for lost time.
Candidates who arrive late may complain about not getting the same detailed instructions as those who arrived on time. My recommendation is to establish a position on tardiness in writing and include it with any other test information provided to candidates. Typically, the best practice for candidates arriving late is to allow them to attend another session, if such sessions are scheduled. Otherwise, set an absolute cut off time and stick with it.
Although I have never had a situation where a candidate had to leave a test session due to illness, it is a good idea to have a plan in case this happens. I recommend handling this situation the same way as someone who is late, which is to schedule them for a subsequent session if one exists and deduct the time they originally spent on the test from the total time allowed, giving them an equal amount of time to finish as the other candidates.
Even when adequate lighting, a comfortable room and proper seating are provided, the best laid plans of test administrators can still go awry. Unforeseen disruptions can include power outages, fire drills and computer failures. Since these types of events are predictably unpredictable, it is important to have a plan for handling disruptions and to include these plans in the instructions for test taking.
Creating comprehensive test instructions is a complex process. Instructions for test taking should be thorough in explaining the use of computerized scoring and proper completion of Scantron forms if they are used. It is critical that instructions for using Scantron sheets include detailed information on how to complete the identification section of the form along with information about the sensitivity of the sheets. Candidates need to know that they should make clean dark lines, avoid random marks, and erase cleanly. Avoiding making marks too dark to begin with facilitates erasures and candidates should know that erasures that are not complete will be picked up as a second answer to an item and items with two answers are scored as wrong.
Test orientation should provide information regarding the locations of restrooms and drinking fountains, the number of individuals allowed to be out of the room at one time and how use of those facilities will impact the time test takers have for completing their tests. Evacuation routes from the test room should also be provided along with regulations regarding the use of cell phones.
As already suggested, the instructions should include what is to be done in case of emergency. These instructions should tell test takers where to go and where to stay along with what they should do with their test materials, which are usually left on their desks. How the jurisdiction handles disruptions in regard to test administration also need to be included in the instructions. Usually, the major concern is how test administration time will be handled. Second to that is how an individual’s response to the disruption might impact that individual’s test performance or his perception of his ability to perform. Since we know that not all individual’s will react the same to a given situation, it is important to make allowances for individual differences in providing options regarding an interrupted test session.
It is easy to add ten minutes to the test administration time if that is the amount of time taken away from a test group in complying with instructions for a fire drill. Disruptions that are longer and have the potential for negatively impacting the group as a whole may require that the group be rescheduled. Test proctors should document disruptions and the time involved so that they can make accurate calculations regarding the time provided to complete the test. Depending on the time of a disruption, the entire session may need to be rescheduled and it will be a judgment call as to whether or not candidates pick up from where they left off or everyone starts over. The amount of time elapsed and the time remaining should help make this call. In most cases, if more than a half hour has elapsed, the candidates should pick up with just the time remaining.
On the other hand, even though an interruption may have been brief, dealing with the emotional upset associated with having a test session interrupted may give rise to claims from individuals that they were not able to get back on track. In those instances, candidates should be given an opportunity to reschedule their test and efforts should be made to accommodate those individuals in terms of dates and times. The guidelines above should be used in determining the starting point for someone in this situation. Providing options seems to have been the key in avoiding any negative impact on candidates as well as any collective actions by test takers that experienced a disruption.
At this point, it is also important to distinguish between what any one individual may consider to be fair or unfair and what is required by standardization. Simply put, standardization attempts to ensure that all candidates are provided the same environment and same opportunity for optimum performance on a test. Fairness, on the other hand, deals with perceptions held by individuals. Such perceptions include things like; the test wasn’t fair because it had too much grammar, or the test wasn’t fair because someone didn’t finish, or the test wasn’t fair because someone signed up late to test, or someone didn’t have enough time to study, or someone didn’t understand the instructions.
Hopefully, you can see that these are individual issues and they should not distract from the point that everyone who took the test had the same amount of grammar on their test, everyone was given the same amount of time to take the test, sign ups were provided on a first come first served basis and study was left up to individuals. I have found that being able to distinguish between standardization issues that need to be addressed and individual concerns regarding fairness helps a test administrator have peace of mind regarding the standardization of their test administration procedures.
