Getting Started

Assessment Basics

1. What is a predictor?
2. What is a KSAP?
   
3. What does the word assessment mean?
   
4. What is an assessment center?
5. What is a biodata instrument?
6. What is a situational judgment test?
7. What is a video test?
8. What is psychometric analysis? 
9. What is validation or validity?
10. What is a job analysis?
11. What is criterion-related validity?
12. What is content-related validity?
13. What is a SME?
14. What is a SME review?
15. What does transportability mean?
16. What are comparable forms?
17. What are equivalent forms?
18. What are parallel forms?
19. What is adverse impact?
20. What is the difference between cognitive ability and non-cognitive abilities?

About Our Tests

1. What kinds of training and experience do your tests require?
2. How do I choose a test?
3. What is a technical report and why is it important to read it?
4. What is a test response data report and what can I use it for?
5. Do you provide test translation services?
6. What are the similarities and differences between your entry-level police officer tests?
7. What are the similarities and differences between your entry-level firefighter tests?
8. What is a TIP?
9. Do you offer study guides?
10. What is a reading list and why is it necessary?
11. Why do you offer both Agency Reading Lists and Candidate Reading Lists?
12. Do you offer fully customized tests?
13. Can I tailor one of your stock tests if particular test questions are irrelevant?
14. What should I do if a candidate has special needs that require my agency to make test accommodations?
15. How do I decide the cut score (or the passing point) for the test?
16. Can candidates bring their electronic devices into the testing room?
17. Are there any special instructions for using Scantrons?
18. What should I do if a candidate wants to review or challenge a test question?
19. Why do you scramble tests?

Ordering Our Tests

1. What is a TSA?
2. Why do you charge an administration fee?
3. What information should I have prepared before I place an order?
4. How do I place an order?
5. What are your shipping options? How long until I receive my tests?
6. What should I do when I receive my order?
7. What should I do with unused test booklets after my test administration?

What is a predictor?

A measurable characteristic used to predict job performance.

What is a KSAP?

KSAP stands for Knowledge, Skills, Abilities, and Personal Characteristics. Examples of KSAPs include Knowledge of Basic Math Calculations, Skill in Welding, and Oral Communication. KSAPs are a type of predictor used to predict job performance.

What does the word assessment mean?

An assessment is any test or procedure used to measure a candidate’s qualifications, which usually includes his or her KSAPs.

What is an assessment center?

A method for assessing a candidate’s KSAPs and performance using an assortment of interactive, simulation exercises.

What is a biodata instrument?

A selection tool containing background data items in which individuals are asked to recall and report typical behaviors or experiences in a situation likely to have occurred earlier in their lives. Biodata instruments may be used to make personnel selection decisions as well as to identify events that lead to destructive tendencies. Overall, biodata instruments are highly reliable, have acceptable to high levels of validity, and have less adverse impact against minority groups than most other instruments used to predict job performance.

What is a situational judgment test?

A test presenting scenarios focused on the tasks and behaviors inherent in the target job. Each scenario depicts a work-related situation requiring the candidate to choose the best course of action by responding to multiple-choice questions.  Candidates must respond using their judgment, logic, and common sense. Responses are scored according to relative level of effectiveness as previously determined by a panel of subject matter experts.

What is a video test?

A video test is a work sample test in which vignettes of on-the-job situations are depicted in video format on a screen or computer monitor. The candidate is asked a series of questions about how they would react to or handle the situation vignettes. 

What is psychometric analysis?

Psychometric analysis is rigorous statistical analysis of test items to ensure that items accurately and consistently measure knowledge and skills.

What is validation or validity?

Validation is the process of determining and documenting what a test or selection procedure measures and the relationship between scores and job performance. Validity is the degree to which actions or inferences based on test results are meaningful or supported by theory and empirical evidence.

What is a job analysis?

A job analysis is a systematic process used to identify the tasks, duties, responsibilities, and working conditions associated with a job as well as the KSAPs required to perform a job.

What is criterion-related validity?

Criterion-related validity is demonstrated by empirical data showing that the selection procedure is predictive of or significantly correlated with important elements of job performance.

What is content-related validity?

Content-related validity is the extent to which the content of a test represents the subject area or behavior it is intended to measure and is related to the requirements and qualifications important for successful job performance. Each of our entry-level tests is based on both content and criterion-related validation studies, while promotional tests are based solely on content validation studies.

What is a SME?

SME stands for Subject Matter Expert. A SME is an individual who is considered to be extremely knowledgeable about the target job and participates in one or more phases of the selection process. Typically, SMEs are incumbents, supervisors, or trainers in the job classification for which the selection process is conducted.

What is a SME review?

A SME review is a Subject Matter Expert Review. In the context of test development, a SME review usually consists of first having SMEs individually rate the importance of the test questions and then having a consensus review with all of the SMEs where they are asked to come to an agreement on the importance of the test questions. SMEs are encouraged to provide feedback to ensure the questions are relevant, that the distracters are not too easy to distinguish from the correct answer, that the language used is clear, etc. Test developers use this feedback to modify the test questions.

What does transportability mean?

Transportability is a specific process whereby evidence of validity may be generalized from one situation to another.

What are comparable forms?

Comparable forms are two or more versions of a test that measure the same KSAPs for which statistical similarity has not been demonstrated. Although these tests contain questions that measure the same KSAPs, the actual questions are different. Agencies may alternate between comparable forms during different test administrations to increase test security.

What are equivalent forms?

Equivalent forms are two or more versions of a test that can be equated by the use of a scoring formula so that scores on each version can be directly compared. Although these tests contain questions that measure the same KSAPs, the actual questions are different. Agencies may alternate between equivalent forms during different test administrations to increase test security. Statistical adjustments can be performed if Agencies want to use the tests as parallel rather than equivalent forms.

What are parallel forms?

Parallel forms are two or more versions of a test that are statistically equivalent in terms of raw score means, standard deviations, error structures, and correlations with other measures for any given population. Parallel forms can be used if Agencies want to deliver two different tests during the same test administration. Although we statistically equated tests that are considered to be parallel forms, the Agencies using these tests as parallel forms should review the technical report to determine whether adjustments are necessary.

What is adverse impact?

The Uniform Guidelines on Employee Selection Procedures defines adverse impact as:  "A substantially different rate of selection in hiring, promotion, or other employment decisions which works to the disadvantage of members of a race, sex, or ethnic group" (Equal Employment Opportunity Commission, Civil Service Commission, & Department of Labor and the Department of Justice, 1978; hereafter, the Guidelines, Section 16B). 

The determination of adverse impact is simply a comparison of the proportions of protected group members (minorities in the various race, color, religion, sex and national origin groups) and majority group members that perform at some specified level on the test.  If the protected group proportion is significantly smaller, adverse impact exists.  Adverse impact simply represents a numerical differential of passing rates and does not automatically imply discrimination or discriminatory intent.  It does, however, indicate the possibility of discrimination.  In that case, whether the test can be considered a business necessity, the validity of the test and the transportability of its use would be reviewed by the courts if the test were challenged.  The Uniform Guidelines state, "If adverse impact exists, it must be justified on grounds of business necessity.  Normally, this means by validation which demonstrates the relation between the selection procedure and performance on the job" (Equal Employment Opportunity Commission, Civil Service Commission, & Department of Labor and the Department of Justice, 1978; hereafter, the Guidelines, Section 10A).

There is considerable debate as to what constitutes a significant difference in the minority/non-minority proportions.  In most cases, the 80% rule of thumb (also referred to in the Uniform Guidelines, as the four-fifths rule) is applied.  The use of 80% as the adverse impact point is the most accepted rule of thumb, both legally and professionally.  As with any rule of thumb, in actual situations a smaller or larger difference in pass rates may be required for a judicial finding of adverse impact. For more information, please order a copy of our handout Considerations in Addressing Adverse Impact.

What is the difference between cognitive ability and non-cognitive abilities?

General cognitive ability is the ability to think and process information. These abilities affect job performance through the role they play in the acquisition and use of information about how to do the job. Some examples of general cognitive processes include understanding, reasoning and intellectual capacity. It’s been thought that individuals with higher levels of cognitive ability acquire new information easier and faster, and are able to use that information more effectively.

Not surprisingly, research has shown that cognitive ability is the best predictor of job performance. Still, it doesn’t tell you everything you need to know about a candidate. What does cognitive ability tell you about how diligently someone works or their interest in the work itself? Cognitive ability ignores these factors, yet conscientiousness and interest in work have been shown to be important predictors of job performance.

Conscientiousness is an example of a non-cognitive ability. Using non-cognitive measures, such as biodata and interest questionnaires, can give you a fuller assessment of each candidate and help reduce adverse impact. Non-cognitive factors include affective, personal, and social variables that predict successful job performance. Examples include interpersonal skills, emotional intelligence, cooperativeness, sense of responsibility, service-mindedness and conscientiousness.

A number of our tests include non-cognitive components. The PO-EL 200 Series includes a 30 question section titled “Interests” that will easily allow you to incorporate non-cognitive testing into your hiring process. For entry-level firefighter positions, both the FF-EL 201-NC and the FF-EL 301-NC include a 20 question non-cognitive section. Additionally, we offer two cost-effective biodata questionnaires: The PO-BDQ 201-NC for police and CO-BDQ 201-NC for corrections.

You may be wondering to yourself, how can you use a non-cognitive measure if you need to stick to the results of your job analysis? Well, are you sure that your job analysis does not indicate that interpersonal skills, emotional intelligence, cooperativeness, sense of responsibility, service-mindedness and conscientiousness are important for success? If you’re sure that these skills and attributes are not important, then you shouldn’t use a non-cognitive measure. However, unless you’re testing for the job of a robot, these factors will be important for successful job performance.

When choosing your tests, remember that it takes many attributes to perform on the job effectively, not just cognitive ability. The more you can assess about the person in advance, the better picture you’ll get for how they’ll perform on the job.

What kinds of training and experience do your tests require?

No prior training or experience is assumed of candidates taking any of our entry-level exams. Some of our entry-level exams require the administration of a TIP before the actual test booklet is distributed. Our promotional exams, however, do require training and experience. They even are accompanied by reading lists.

How do I choose a test?

Customers frequently call us and say, “You have several tests for the same rank. How do I choose one?” While we can’t tell you which test to administer (yes, we know your life would be easier if we could), we can tell you how to decide which test to use.

What do the results of your job analysis say? It is critical to base the selection of your test on the results of a job analysis. A job analysis is a systematic process used to identify the tasks, duties, responsibilities, and working conditions associated with a job and the knowledge, skills, abilities, and personal characteristics required to perform a job. You might be thinking to yourself, “Don’t you guys perform your own job analyses? Why can’t we just use yours?” Well, you can and you can’t.

We have performed our own job analyses during the development of each test, but it is your responsibility to ensure that the position you are testing for is the same as the position studied during the development of the test. For entry-level tests, you want to ensure you are choosing the right test for the job and that you meet legal requirements.   When it comes to promotional tests, you want to consider that different departments may have different requirements. In a smaller police department, for example, there are fewer staff members and, as a result, different ranks may be may be responsible for a greater variety of tasks. In smaller departments, Captains may be responsible for budgeting tasks, while in larger departments Lieutenants may be responsible for this duty. 

This is why you want to make sure that the test you are selecting is assessing KSAPs that are necessary for your job. This is not so much the case with entry-level tests, but it is especially true for promotional tests.  You need to make sure that the job we’re calling entry-level police officer, for example, is the very same job you are talking about and that the areas we say are important for successful job performance are the same ones that are important for success in your locality.

How can you do this? Let’s say you’re thinking of using the PO-EL 101, which is one of our entry-level police tests. You would then need to compare the results of your own job analysis for entry-level police officer with the job analysis that was done for the development of the PO-EL 101. This can also be done by conducting a transportability study, which you can do yourself or use our transportability study service (TSS).  The Transportability Study Service (TSS) assists IPMA-HR test users in linking the original development and validation study to the job which the user wishes to apply the test. The results of a transportability study would tell you whether the KSAPs assessed by the test are substantially similar to those necessary for success on the job.

We swear that we’re not trying to sell you stuff; it’s actually the law. The Uniform Guidelines on Employee Selection Procedures states that a thorough job analysis is needed for supporting a selection procedure and, if you are using a published test, you must link the original development and validation study to the job for which you want to use the test. And if you don’t know what the Uniform Guidelines are, uh, you really should. They are available for free at http://www.uniformguidelines.com.  The cut score handout and our Test Administration Handbook also discuss them.

We’ve digressed. So, you’ll need to figure out if you have an up-to-date job analysis or whether you want to conduct a transportability study. Once you’ve decided those things, you’ll need to get your hands on a copy of the original job analysis and validation study. Where can you find the job analysis that was done for any of our tests? You can always find the results of each test’s job analysis in that test’s technical report. The technical report is that heavy document you might be using to prop your door open. No, it’s not that thick, but it does take some time to read. Once you’ve read it though, you will have a better understanding of how the test was developed. Technical reports are complimentary and are available to those with a TSA on file.

Once you’ve compared the results of your job analysis to ours or conducted your transportability study, you will be in good shape to choose a test. Just pick one that assesses the areas your job analysis or transportability study showed as being critical for success. 

What is a technical report and why is it important to read it?

A technical report provides information on the job analysis, test development, and validation process for a given test. A security agreement is necessary to order technical reports.

What is a test response data report and what can I use it for?

A test response data report presents test data gathered on candidates from jurisdictions that use IPMA-HR tests. This includes information about passing points, score distributions, and adverse impact.

Do you provide test translation services?

IPMA-HR's Assessment Services Department will make every attempt to provide translation services for our test products if translation is required by law. However, this service is subject to the availability of translators and foreign language proof readers. In the past, we have only provided translation services if translation is required by law. For example, it makes sense to translate one of our English exams into French if you work in Quebec where French is the official language.

If your agency would like to request a translated test, please understand that we cannot endorse your decision unless the translation is required by law. For this reason, you may want to consult with your agency's legal department to discuss the necessity of creating a translated test. If you decide to special order a translated test, we will provide you with a quote for this service. If you have further questions, please contact Andrey Pankov.

What are the similarities and differences between your entry-level police officer tests?

All entry-level police officer tests are based on the same job analysis and are multiple-choice tests with at least 90 questions. These tests, however, differ across several dimensions including their content areas and questions, the exact number of test questions, format (video or traditional paper-and-pencil), and whether there are alternate forms and supplementary materials available.

	TEST TITLE
	PO-EL 101 (TIP)	PO-EL 102 (TIP)	PO-EL 201-NC (TIP)	PO-EL 202-NC (TIP)	PO-EL 203-NC (TIP)	PO-EL 301 (VID)
Ability to Learn and Apply Information	Yes	Yes	Yes	Yes	Yes	Yes
Ability to Observe and Remember Details	Yes	Yes	Yes	Yes	Yes	Yes
Verbal Ability	Yes	Yes	No	Yes	Yes	No
Ability to Follow Directions	Yes	Yes	Yes	No	No	No
Ability to Use Situational Judgment and Solve Problems	Yes	Yes	Yes	Yes	Yes	Yes
Police Interest Questionnaire	No	No	Yes	Yes	Yes	No
Format	Paper	Paper	Paper	Paper	Paper	Paper and Video
Test Time Limit (Hours:Minutes)	1:45	1:45	1:45	1:45	1:45	2:35
TIP Time Limit (Hours:Minutes)	0:25	0:25	0:25	0:25	0:25	N/A
Number of Questions	100	100	100	100	100	90

What are the similarities and differences between your entry-level firefighter tests?

All of our entry-level firefighter tests are also based on the same job analysis and are multiple-choice tests with at least 80 questions. These tests, however, differ across several dimensions including their content areas and questions, the exact number of test questions, and whether there are alternate forms and supplementary materials available.

	TEST TITLE
	FF-EL 101	FF-EL 102	FF-EL 201-NC (TIP)	FF-EL 202 (TIP)	FF-EL 301-NC (TIP)	FF-EL 302
Ability to Learn and Apply Information (TIP)	No	No	Yes	Yes	Yes	Yes
Interests	No	No	Yes	No	Yes	No
Logical and Mathematical Reasoning Ability	Yes	Yes	Yes	Yes	Yes	Yes
Mechanical Aptitude	Yes	Yes	No	No	No	No
Reading Comprehension	Yes	Yes	Yes	Yes	Yes	Yes
Situational Judgment	Yes	Yes	Yes	Yes	Yes	Yes
Spatial Sense	Yes	Yes	No	No	No	No
Vocabulary	Yes	Yes	No	No	No	No
Format	Paper	Paper	Paper	Paper	Paper	Paper
Test Time Limit: (Hours:Minutes)	2:00	2:00	2:00	1:45	2:00	1:45
TIP Time Limit (Hours:Minutes)	N/A	N/A	0:20	0:20	0:20	0:20
Number of Questions	90	90	100	80	100	80

What is a TIP?

The acronym TIP stands for Test Information Packet. Each TIP is designed to help assess candidates’ ability to read, learn and apply information of the type they will be required to utilize on the job. Each test has its own unique TIP. That is, there is one TIP for the PO-EL 101 (TIP), one TIP for the PO-EL 102 (TIP), and so forth. TIPs are an essential, timed part of the tests they accompany and are distributed and collected just prior to handing out the test booklets. A certain percentage of questions in the test booklet will ask candidates to recall information they studied in TIPs, in order to assess how well they were able to learn and absorb the material. Candidates are NOT allowed to take notes on the TIP for use during the test, but they may highlight information as they read through the TIP. Once the time limit ends, TIPs should be collected. At this point, the test administrator should then distribute the actual test and scratch paper.

Do you offer study guides?

Yes, we do offer study guides for our tests. Currently, we have three study guides available: one for the PO-EL 100 Series, one for all of our entry-level firefighter tests, and one which covers all of our entry-level police officer tests. 

 What is a reading list and why is it necessary?

A reading list is a list of books used by candidates to prepare for promotional tests. Typically, our reading lists contain approximately 5 to 6 books. Although we aim to minimize this number, it's difficult to find books that assess all of the KSAPs that the job analysis showed are crucial for success on the job. Decreasing the number of books would most likely impact the validity of the test. To some extent, the smaller the range of books, the less the test will be able to predict success on the job. 

Occasionally, we hear rumors that our customers don’t distribute our reading lists to their candidates. We cannot force you to distribute the reading lists to candidates, but we can tell you that we would personally distribute the reading lists to candidates if we were administering the tests, and here’s why. While candidates may know the answers to some test questions based on their job experience, other questions talk about specific information that they might not get from job experience alone.

You might be thinking that we’re trying to teach to the test, but you would be wrong. The content for any one of our promotional tests is based on the results of job analysis and content validation study. We simply created questions that subject matter experts like you told us were important for successful job performance.

If you have reviewed the test, compared our job analysis to yours and you agree that everything is relevant, candidates need to study those books. Our promotional exams are, after all, knowledge-based tests. Unlike our entry-level exams, our promotional exams do require training and experience. Part of this training involves a study period, which, in itself, is an added benefit for the department. Why wouldn't you want your candidates to read all that job relevant, knowledge-rich material? 

We know that the books are expensive and take a lot of time to read, but just remember that how well candidates do on the test is a indication of their commitment to the job and strong indication of how well they’ll do on the job. If cost is an issue, however, you might suggest that small groups of candidates purchase the books together. They can then circulate the books among themselves. This would cut down on the cost tremendously. 

We require that the Candidate Reading List be distributed to candidates for study purposes. We strongly recommend that the candidates have 60 days or more to study the books before they take the test, allowing an extra one to two weeks for candidates to obtain a copy of each book. It is important to remember that reading lists should not be posted year-round. Doing so would provide applicants with an unfair advantage. This would, in turn, invalidate the test results.

Finally, since new editions of books are published or go out-of-print, it is very important that you contact us before posting a reading list.

Why do you offer both Agency Reading Lists and Candidate Reading Lists?

Previously, we only distributed reading lists that were intended for use by the agency. However, some agencies distributed these lists to their candidates and, in turn, the candidates contacted us with questions about the test process. As stated in the TSA, candidates may NOT contact us directly. To resolve this issue, we created two reading lists. The Agency Reading List should be used by authorized personnel for review purposes, while the Candidate Reading List should be distributed to candidates. At no time, should candidates be encouraged to contact us directly. Please also note that, while you may distribute the reading lists to candidates, under no circumstances may you post reading lists or their contents on your website. Reading lists are secure material and may not be shared with the public.

Do you offer fully customized tests?

Yes, our Customized Test Service enables agencies to create tests using their own job analysis information. Customization allows agencies to assess competencies that are important for effective performance at different ranks within their departments.

Currently, you can create customized tests for the following public safety and non-public safety positions: Fire Captain, Fire Assistant Chief, Fire Battalion Chief, Fire Chief, Police Captain, Police Major, Police Assistant Chief, Police Deputy Chief, Police Chief, and Generic First-Line Supervisor (A generic first-line supervisor test can be used for those hard to fill first-line supervisory positions throughout your agency).

We also offer agencies the option to create 25 specialized questions from one or more documents or texts of their choice. Specifically, our Assessment Services Department offers agencies the opportunity to incorporate specific texts and internal documents that their department has created such as city ordinances, departmental policies, rules, regulations, and standard operating procedures into their semi-stock tests.

If any of this sounds attractive, the best thing for you to do is to order a Customization Packet for Police, Fire, or Generic First-Line Supervisor. The customization packet will break down the customization process for you and even includes a time line for planning purposes. By ordering this packet, you are under no obligation to order a customized test.

If you are interested in a customized test, please be aware that you’ll need to plan ahead at least three weeks in advance depending upon your specific needs. It’s possible to order customized tests fewer than three weeks in advance, but it's not encouraged. The less time we have to communicate with you decreases the chances of you getting exactly what you want. For questions about Customized Tests, please contact Jacob Jackovich.

Can I tailor one of your stock tests if particular test questions are irrelevant?

Our Customized Test Service enables agencies to customize promotional stock tests to create something called a semi-stock test. A semi-stock test is a unique blend of questions that appear in our promotional stock tests and of items that do not appear in the stock tests. Test questions can be added or deleted from the stock tests. Adding questions to the stock test can be done in one of two ways—by selecting questions from our item bank or ordering new questions through our Item Writing Service. Customization allows agencies to assess competencies that are important for effective performance at different ranks within their specific departments. Please note that while semi-stock tests are available for any of our stock promotional exams, additional questions for corrections and emergency communications tests are only available through our Item Writing Service. If this option interests you, the best thing for you to do is to order a Semi-Stock Test Packet.  For questions about Semi-Stock Tests, please contact Jacob Jackovich. 

What should I do if a candidate has special needs that require my agency to make test accommodations?

Since we do not have standard procedures to which your agency must adhere, consider perusing the Americans with Disabilities website. Although we have not established standard procedures, federal and state laws require that the candidate show proof of disability (e.g., a doctor's note stating the disability). If possible, you may want to consider testing this candidate separately from the other candidates so as to avoid disruption.

You may also want to consider extending the time limit of the test. You may want to provide at least one-and-a-half times the standard time limit and a more cautious approach would be to allow twice the time limit. Additionally, some agencies rely on the use of readers to recite the test and test information packet aloud to candidates. If you choose to employ a reader, please factor this decision into how much time you have allotted the candidate to take the test. That is, using a reader may extend the total time of the test. Usually, state-level agencies employ readers or have lists of trained readers. A friend or relative is usually a bad choice.

We offer a handout on testing accommodations, Considerations in Test Accommodations. This handout is a resource for HR professionals who need assistance in making test accommodations for applicants with disabilities. The handout represents a composite of information from the professional and legal guidelines related to personnel selection. These include the Americans with Disabilities Act, Principles for the Validation and Use of Personnel Selection Procedures, Standards for Educational and Psychological Testing, and Uniform Guidelines on Employment Selection Procedures. 

How do I decide the cut score (or passing point) for the test?

Setting cut scores is a complicated process, which (unfortunately) requires some in depth thought and analysis. Did you know that there were multiple methods that can be used to select a cut score? And, of course, each cut score setting method has its benefits and drawbacks. Whichever method you choose, it is important that you document your reasoning for choosing it. Although we cannot recommend cut scores (boy, wouldn’t life be easier if we could?), we do provide a handout that talks about how you should pick one. In 20 pages, this handout answers the following questions:

• What do the law and professional guidelines say about setting cut scores?
• What legal precedents should you be aware of?
• Which individuals are qualified to serve as SMEs on the panel that will assist in setting the cut score? 
• How do the individuals who are selected to serve on the SME panel feel about the selection instrument?
• Are the SME panel members diverse enough so that the applicants are adequately represented?
• How many individuals are needed to serve on the SME panel?
• How will the SMEs be trained?
• How much interaction should SMEs have during the cut score setting process?
• How many rounds of deliberation are needed to make valid recommendations?
• How should recommendations be combined across raters?
• How should the cut score setting process be documented?
• What data can be used to determine an appropriate cut score?
• Can various methods be integrated and used to set the cut score?
• Are different cut scores needed for different parts of the selection or promotion process?
• How complex is the selection tool for which the cut score is being set?
• What methods can be used to set cut scores?

If you want to read another 20 pages or so, you can also get some information about test score banding and the controversy surrounding its use. However, to make an informed decision about cut scores, you only need to read the first 20 pages. You may be thinking that 20 pages is still a lot. Maybe you’re even wondering, “Why can’t I just call IPMA-HR for a summary?” Well, unfortunately this handout is the summary. As we said, setting cut scores is a complicated process. And, by complicated, we just mean that it takes some good old fashioned in depth thinking. Some research has to be done, and some decisions need to be made. Just think, the more time you take selecting your test and cut score, the more confident you will feel during the test’s administration. 

The bottom line is that we know you’re busy (we are too!), and it can be very tempting to cut corners. In today’s economy, the job market is tight and law suits are on the rise, especially in public safety. The selection of your test and cut score are not places to cut corners.

If you have read our handout and don’t understand what you read, consider speaking with someone in your organization who has a background in statistics, HR, test development, or I-O Psychology and kindly ask for their help. Whatever you do, please do not select 70% as your passing score without a substantial reason. And let us be clear that, “We’ve always used 70%” is not a good enough reason.

While we hope that you never find yourself embroiled in a law suit, if you do, will you wish that you had more confidence in the test and cut score? If the answer is yes, then please order your free copy of the handout today. It’s called Considerations in Implementing Selection Procedures.

Why do you offer both Agency Reading Lists and Candidate Reading Lists?

Previously, we only distributed reading lists that were intended for use by the agency. However, some agencies distributed these lists to their candidates and, in turn, the candidates contacted us with questions about the test process. As stated in the TSA, candidates may NOT contact us directly. To resolve this issue, we created two reading lists. The Agency Reading List should be used by authorized personnel for review purposes, while the Candidate Reading List should be distributed to candidates. At no time, should candidates be encouraged to contact us directly. Please also note that, while you may distribute the reading lists to candidates, under no circumstances may you post reading lists or their contents on your website. Reading lists are secure material and may not be shared with the public.

Can candidates bring their electronic devices into the testing room?

Cell phones are not just for making phone calls anymore. They can be used for text messaging, web browsing, taking pictures, and listening to audio. Even the most basic of cell phones are powerful communicative devices capable of a wide range of tasks. The next time you administer an exam, please keep these multifunctional devices in mind when thinking about what items candidates should and should not bring into the testing room with them.

We’ve become aware of cheating rings where candidates have collaborated to cheat on a test by transmitting text messages to one another either while sitting in the testing room or from an area outside the testing room (e.g., during a restroom break).

Even working alone, candidates can easily record notes to any number of electronic devices (e.g., cell phones, iPods) before the test and then discreetly listen to that information while taking the exam. Before you say, “Well wouldn’t I see the earphone wires?” These days, there are extremely small, in-ear listening devices available, which can be used to wirelessly listen to material. So, unfortunately, the answer is no; you may not notice any wires because there might not be any!

Candidates should be notified in advance that cell phone, iPods, and other similar electronic devices are prohibited from the testing room. If candidates arrive with such devices, consider asking them to leave those devices outside the testing room (e.g., in a car or a locker) or check them with a proctor prior to entering the testing room. Simply asking candidates to turn off their cell phones and other electronic devices is not a sufficient security measure, as it is possible to surreptitiously activate the device without notice, especially during a busy time of the administration when proctors are otherwise engaged.

Even if you allow cell phones into the testing room, it is extremely important that proctors be on the lookout for questionable and repetitious behavior during the test administration. Repeated gestures, such as reaching into a pocket or cupping a hand over an ear, may indicate that candidates are receiving outside communication to gain an unfair advantage on the test. Careful observation of candidates by proctors during test administration is a powerful deterrent for prospective cheaters.  For more helpful tips, check our Test Day Administration Guide.

Are there any special instructions for using Scantrons?

If you are using our electronic scoring service then candidates will use Scantrons as their answer sheets. If you are using Scantrons, tell your candidates they must erase their answers thoroughly if and when they decide to change their answers. Why is this important? If Candidate Sally Ricks does not erase her original answer thoroughly enough and marks another choice, even if she marks the right choice this time, the computer will read her erasure as an invalid response and she will not receive credit for her answer.   Make sure to emphasize the importance of erasing thoroughly to candidates before they take the test. 

If there is a problem related to erasures and/or invalid responses, the computer will mark the answer with an asterisk (i.e., *). As a secondary precaution, you could review the item analysis report for asterisks and check the candidate answer sheets to make sure candidates received proper credit.  

What should I do if a candidate wants to review or challenge a test question?

We have a complimentary Item Challenge Service for promotional stock tests, customized tests, and semi-stock tests. We offer a handout called Considerations in Handling Item Challenges that contains detailed steps for resolving item challenges while still ensuring test security. If you have already ordered your copy and have questions regarding this handout, please contact Andrey Pankov.

Item challenges of our entry-level tests are extremely rare. However, if you find yourself in this situation, contact Andrey who will research and confirm the accuracy of the disputed question(s) and answer(s). If the candidate pursues the complaint further, you may tell the candidate that the test has been validated and that it is supported by an extensive technical report. Please do not, however, show the candidate the technical report. Doing so would represent a violation of your TSA. If the candidate is still not satisfied, the only way for the candidate to actually see the test would be for the candidate to successfully convince a judge that he/she has just cause for wanting to review the test. Please note that in cases of formal charges of discrimination by the Equal Employment Opportunity Commission, the Office of Revenue Sharing, the Department of Justice, etc., our tests and scoring keys may be released under a protective court order. You MUST contact us prior to the release of these materials as required by our TSA.

Why do you scramble tests?

Periodically, we scramble or reorder test questions to increase test security.

What is a TSA?

No, it’s not the Transportation Security Administration, it’s our Test Security Agreement. This agreement is intended to protect the mutual interests of all public agencies and officials that use test materials obtained from IPMA-HR, as well as the interests of persons who take such tests. In order that no person may gain special advantage by having improper access to the material IPMA-HR requires as a condition for making its test materials available that all users sign this agreement and fulfill its terms.

Why do you charge an administration fee?

This fee covers expenses related to the preparation of test materials.

What information should I have prepared before I place an order?

• Who is the authorized signer on the agency’s account? Test products will only be shipped to the attention of an individual who has signed IPMA-HR’s Test Security Agreement. In the event a TSA is out-of-date at the time of your order, a new test security agreement will need to be filed before the order can be processed.

• Which test(s) would you like to order? Please consult the descriptions of our test products and utilize the free technical reports and inspection copies to decide which test will give you the most accurate assessment of your candidates. See the question above, How do I choose a test?

• How many test booklets will you be ordering? Every candidate you test will require his or her own test booklet. In deciding on the quantity to order, consider the possibility of late applicants and various other unforeseen circumstances. In the event you do not have a definite number of candidates, consider ordering additional test booklets, which are refundable (minus a $3.50 restocking fee per booklet) if the booklets are returned unused within 2 weeks of your scheduled test date.

• How would you like to score them? We're committed to providing you with simple and secure options to score your tests. Three options are available:

  1. Hand scoring: Hand scoring stencils are available for each test ordered. Your first stencil is available free of charge. Extra stencils may be rented for an additional cost of $10.00 per stencil.

  2. Answer Key: Agencies with access to an optical scoring machine, such as a Scantron, may request an answer key. IPMA-HR will not provide answer sheets for your candidates, but you should have access to sheets that are compatible with your machine. You may use the key we provide to prepare the machine for scoring.

  3. Scoring Service: Computerized machine scoring can be obtained through our Scoring Service, provided by an off-site vendor. An item analysis of the test results is provided to agency’s requesting this method of scoring. There is an additional fee for this service of $40.00 per test title, plus $0.50 per answer sheet.

• What is your scheduled test date? Providing your test date helps us plan your shipment to ensure that you receive your tests with ample time before administration. Additionally, if you anticipate having extra tests after your administration that you will return for credit, a test date is required for your order to be eligible for return.

• How do you want your materials shipped? All secure test products must ship to the attention of a TSA signer. Free shipping for most test products is available to all customers in the continental United States (via UPS Ground) and to Hawaii and Alaska (via USPS Priority Mail). Rush shipping is available at an extra cost of $20 plus 10% of the order total. Please contact the Assessment Services Department for international shipping options.

How do I place an order?

Orders for test material may be placed in a variety of ways:

• Internet: You can order online at http://www.ipma-hr.org/assessment. Browse to the page of the product you'd like to order and added the requested quantity, scoring method and test date for that product to the shopping cart. Fill out all of the order information and submit. We will confirm your order within 24 hours by e-mail to the address provided.
• E-Mail: Contact us with the relevant details for your test order via e-mail at assessment@ipma-hr.org. An e-mail response with order confirmation will be received within 24 hours.
• Phone: Call us at (800) 381-TEST (8378). Orders may be placed by phone during regular business hours and an order confirmation number will be provided immediately.
• Fax: Fill out the test order form and fax the order to (703) 684-0948 (Attn: Assessment Services Department). Confirmation of faxed orders will be received within 24 hours via either the provided phone number or e-mail address.
• Mail: Fill out the order form and mail to:
  IPMA-HR
  1617 Duke St
  Alexandria, VA 22314.

Please allow ample time for the delivery and processing of your order form to take place. Confirmation of your order will be provided via e-mail or phone once it is processed.

What are your shipping options? How long until I receive my tests?

We offer two basic shipping options: UPS Ground, which is at no extra cost, or UPS 2nd or Next Day, for which we charge a rush shipping fee of $20 plus 10% of the order total.

We make every effort to ship test materials the following business day from when you place your order. In situations where you order very far in advance of your test date, we may hold your shipment date back until approximately 2 to 3 weeks before your test date.

If we have an email on file for the Test Security Agreement signer we are shipping your order to, then an automatic shipment notification will be generated and emailed to that signer when we process your shipment.

All of our outgoing shipments originate from our office in Alexandria, VA. UPS Ground shipments will vary in delivery time from 1 to 6 business days, depending on where you are located. Please consult the following UPS Ground map to get a general idea of how long a UPS Ground shipment will take to get to your location (current as of 5/2/2011, but subject to change without notice):

What should I do when I receive my order?

Only an authorized signer with the receiving agency should open any package from us that contains secure test products. A warning on the exterior of the package will make you aware of the security of the enclosed products. Upon receiving the package, an authorized signer should examine the contents of the shipment to ensure that all requested materials are enclosed. We make every effort to ensure that all orders are filled accurately and arrive in time for the test to be administered.

Do not wait until the day of the test to check your order. It is the responsibility of the test-utilizing agency to verify that all requested materials are present prior to the test date. If any expected items are missing from your package, please contact us immediately.

Once the package is examined and everything is found to be in order, a security procedure should be implemented and followed for the duration of the rental agreement to ensure the tests remain secure. Materials enclosed with every order can assist you in securing the tests.

What should I do with unused test booklets after my test administration?

You have two options, both with factors that you should take into careful consideration prior to deciding:

1. Return the unused tests for credit. We will accept returns within 2 weeks of the test date you provided at the time of order, allowing a reasonable grace period to allow for shipping times. Instructions to process your return are included with your order. There is a $3.50 re-stocking fee that applies for each test booklet returned, however. 
2. Keep the tests on hand for future administration.To avoid costly return shipping and incurring the re-stocking fee for returning unused tests, we allow you to hold onto the unused tests you’ve ordered, so long as you comply with the terms of the Test Security Agreement and keep the tests secure. You should also keep any supplemental material (administrator copy, test day administration guide, etc.) that came along with the test, along with scoring materials. If you utilized our Scoring Service, then you will need to pay for additional uses of the service beyond the original order.

WARNING:We make no guarantee as to the future availability of a particular test. We periodically alter tests, sometimes in significant ways (changing questions, etc.), other times in more cosmetic fashion (scrambling the order test questions appear, etc.). If the situation should arise where you need to order additional copies of a test you already have on hand, please contact IPMA-HR beforehand so we can discuss further and find out if you’ll be receiving the same test – or one that’s undergone some changes. In some circumstances, we may request that you review an inspection copy and compare it yourself to the test you have on hand to verify the content is the same. IPMA-HR cannot be held responsible for the use of mismatched tests in these circumstances.