How Our Tests Are Developed
The first thing you should know about our test development process is that it follows very strict professional principles and guidelines set forth by the human resources industry, specifically:
- The American Psychological Association’s Standards for Educational and Psychological Testing.
- The United States Equal Employment Opportunity Commission’s Uniform Guidelines on Employee Selection Procedures.
- The Society for Industrial and Organizational Psychology’s Principles for the Validation and Use of Personnel Selection Procedures.
You can rest assured that every test we develop undergoes a comprehensive and systematic process to ensure its validity. This process relies on two parts: content relevance and criterion relatedness.
Our test development process always starts with a thorough job analysis. This is when we gather information from subject-matter experts (SMEs). SMEs are the men and women who have been doing the job for a number of years — and doing it well — in departments and organizations located all over the country. The information we gather is very important to ensuring that the content of the test is relevant to the job. There are three steps to this process:
- Job Analysis Questionnaire. SMEs complete a job analysis questionnaire, which asks them to rate three things:
- How important specific tasks are to the job.
- How frequently those tasks are performed.
- How important the knowledge, skills, abilities, and personal characteristics (KSAPs) listed are to performing the job effectively.
- Data Analysis. Once all the data has been collected from the questionnaires, we analyze it to determine the most important KSAPs needed to do well in that particular job.
- Test Blueprint. Our analysis provides us with the information we need to create a blueprint for developing test items.
One Step Further
When developing promotional tests, which are rank-specific tests that help you assess the promotional potential of your employees, we take the content relevance part of the development process one step further.
After the tests are developed, we ask a group of SMEs (for promotional tests, they are supervisory and administrative-level employees) to review the test questions, and for each one, rate its clarity, importance and relevance to the job.
In this part of the validation process, which is completed for all our entry-level, multiple-choice tests, we take steps to ensure that test scores directly relate to a candidate’s job performance. There are four steps to the process:
- Gather Test Scores. Once the test is drafted, we ask men and women from departments and organizations around the country, who are currently doing the job, to take the test.
- Gather Performance Ratings. Next, we ask their direct supervisors to evaluate their job performance using a standard form provided by us.
- Test the Test. Then we evaluate the information, facts and figures gathered in steps 1 and 2 to identify patterns — specifically, to ensure that a candidate’s test score does, in fact, predict his or her job performance.
- Item Analysis. Finally, an item analysis is conducted to identify test items that are not working well. For example, test items that are statistically shown to be too easy or too difficult, or that fail to show a difference between those who do well on the test — and those who don’t.
Who Develops Our Tests
Combined, the experts who make up our test development team have over 80 years of experience. They are recognized experts in their field, having received national awards for their work, been asked to speak at numerous conferences and workshops, served as expert witnesses and government advisors.