Oct 1, 2021
by Yari Randall
Pre-employment testing dates as far back as 206 BCE. Today, standardized, valid and reliable pre-employment assessments are widely used in the public sector due to their proven benefits of (1) increasing productivity and employee retention; (2) strengthening the legal defensibility, objectivity and efficiency of hiring processes; and (3) decreasing costs associated with turnover and the time it takes to complete the hiring process.
Assessments can be useful tools for identifying the most qualified candidates for selection. However, as one of my colleagues once said, assessments are like scissors in that they can be particularly harmful if misused or implemented in an ill-informed fashion. One of the risks associated with misusing pre-employment assessments includes unfairly discriminating based on protected traits such as race, color, sex, religion and national origin.
In this column, I will review ways agencies can use assessments successfully to identify qualified candidates and also promote diversity, equity and inclusion.
Assessment Score Differences Between Groups Do Not Disqualify Tests
It is not unusual to observe differences in assessment scores between groups. For example, individuals who are members of historically marginalized groups tend to score lower on assessments of cognitive ability than do members of dominant groups. This appears problematic because cognitive ability assessments are widely used for pre-employment testing, as studies have shown they consistently predict job performance better than any other types of assessment. James L. Outtz discusses this in depth in his 2009 book Adverse Impact: Implications for Organizational Staffing and High Stakes Selection.
Gender differences have also been observed in physical ability and personality assessments.
In the United States, certain groups, including women, African Americans and immigrants historically encountered restricted educational and employment opportunities. Some such disparities in educational and employment opportunities are still apparent today. A 2019 study by the group EdBuild.org found that predominantly white schools received approximately $23 billion more in funding than did predominantly Black schools. Women in the United States still earn only a fraction of what men earn.
While the presence of group differences in assessment scores may be alarming, it does not warrant complete avoidance of assessments. Administering assessments that measure job-related competencies is useful.
Score Differentials Do Not Equate to Adverse Impact
In the landmark case Ricci v. DeStefano (2009), an agency was sued for throwing out the results of an assessment for fear that using the results to make selection decisions would have an adverse impact on historically marginalized groups. Although the assessment had documented score differences between groups, it was professionally validated and measured job-related competencies. The U.S. Supreme Court held that the agency’s actions violated Title VII of the Civil Rights Act of 1964 and that fear of disparate impact liability is not an acceptable basis for throwing out the results of a valid, job-related assessment.
As stated by Highhouse, Doverspike and Guion in the 2016 edition of Essentials of Personnel Assessment and Selection, “Adverse impact is a legal term, not a statistical or psychometric one, referring to whether there are practical or significant differences in the selection ratio when comparing different groups.” Here, it is important to note the mean differences in scores between groups reflect the statistical or psychometric properties of assessments. Using an assessment that has statistical mean differences in scores between groups does not automatically mean that adverse impact (i.e., a significant difference in selection ratio between groups) will inevitably occur.
Methods to Reduce Assessment Score Differences Between Groups
There are legally and professionally supported methods agencies can use to reduce group differences in assessment scores. Options include
- Using a combination of different assessment types (e.g., cognitive, personality, physical) and methods (e.g., written test, structured interview, video-based test). This helps provide agencies with a more complete picture of each candidate’s job-related competencies. Relying on more than one type of assessment also gives an organization the opportunity to strategically weigh scores on each assessment to help reduce overall mean score differences.
- Reviewing assessments for biased or insensitive content. This allows agencies to remove or modify content to ensure questions or scenarios do not negatively impact the way different groups perform. For example, the presence of stereotypical content could induce a phenomenon called stereotype threat. A 2006 article on the American Psychological Association website explains stereotype threat can widen the achievement gap and cause members of the stereotyped groups to perform worse than they would have if the stereotyping content were not present.
- Offering study guides, practice tests or coaching opportunities prior to conducting the assessment. Cognitive psychology research has established practice testing as an effective strategy for improving learning and test-taker performance.
Methods to Prevent Adverse Impact When Using Assessments
There are also legally and professionally supported methods agencies can use to prevent adverse impacts when using assessments. Two that have passed court review are
- Setting a cut score or using performance bands instead of using top-down selection. This prevents agencies from prematurely eliminating otherwise qualified members of marginalized groups from candidate or promotion pools simply because they are not among the top scorers.
- Using the compensatory approach instead of the multiple hurdle approach. Inviting candidates to take all assessments included in a selection battery and combining their scores on each can help prevent agencies from prematurely selecting out members of marginalized groups who do not score as highly on certain “hurdles.”
Assessments have been an integral part of employment selection systems for thousands of years. They are likely here to stay. Like most tools, however, they must be used responsibly and with care. Failing to use assessments in the ways that research indicates are best puts agencies at risk for turning down qualified applicants and inhibiting the diversity of their leadership and overall workforce.