As Diversity and Inclusion has become a key priority, it is time to re-examine accepted assessment and selection practices. Some of our most trusted techniques can be at odds with our aims to attract and select a more diverse talent pool. Driven by time and budget considerations, we have often used a funnel approach with very different tools at each tier to establish what capabilities a candidate could bring to the role; large scale application of ability tests to shortlist, followed by interviews and then perhaps exercises that simulate activities and decision making within the role itself.
1. Ability Tests
If we cannot meet or observe everyone, we might use cognitive tests that can be repeated on a large scale. These measure abstractions of qualities and capabilities, statistically proven to be associated with high performance on the role. They offer a feel of objectivity as the candidate cannot fake their results.
2. Interviews
For a more select group we use interviews to hear where capabilities have been demonstrated in the past. We advise interviewers to avoid hypothetical questions and focus on real life experience to identify what they can bring to the new role.
3. Work Simulations
Where we can afford to, the most compelling evidence is secured by observing candidates carrying out “realistic” samples of the role itself to see first-hand who best demonstrates the relevant capabilities right now within a relevant context.
Diversity Risks through the Selection Process
Using ability tests at the early stages is cheap and seemingly effective. Unfortunately, our comfort with ability tests comes at a cost to diversity. Research proves that abstracted hard ability tests at a sift stage can cause significant adverse impact and remove high quality people before they get the chance to prove themselves with the more realistic assessments later on. They can also negatively impact the candidate experience; feel mechanistic, deny opportunity for human connection and provide little insight about the company or role, creating a threatening environment that might particularly discourage minority candidates.
Interviews create different barriers to diversity. Bias can impact the questions you ask, the rigour with which you explore the answers as well as your interpretation of the answers. A competency-based question schedule can help ensure questions are relevant and consistent for all. However, focusing questions on what candidates have done in the past assumes all have had equal opportunity in the past to exercise those competencies. Unfortunately, we know that stretch opportunities are not experienced equally by all and those in minority groups are further disadvantaged by not having the best relevant past experiences to share.
The more real-life an assessment the lower the level of adverse impact as it gives everyone the chance to show what they “could” do if given the chance. However, even work simulations are not immune from bias. As work simulations are observed, typically they focus on observable capabilities and behaviours through role-plays, group discussions and presentations. Well trained observers will be mindful of the impact on unconscious bias in their observations. However, some “thought- based” competencies are less intrinsically observable, and more subject to our assumptions about a candidate’s thought processes.
How Pearn Kandola use Situational Judgment Tests
Pearn Kandola have been working with clients to utilise the power of Situational Judgement Tests (SJTs) to improve assessment quality through all stages of the selection funnel. SJTs provide powerful, “real life” and often cost effective improvements that diminish adverse impact. This improves the quality of each assessment stage by ensuring all the very best candidates are identified and are not unfairly excluded.
1. For large scale assessment, SJTs can be used online to evaluate critical decision-making needed in the role. The judgements not only reflect candidate reasoning ability but also their values which cognitive tests can miss such as ‘integrity’. The tests are less threatening and more accessible than cognitive tests resulting in lower adverse impact. This can be further enhanced by using video to reduce the need to read too much text and to increase the interactive “real-life” feel of the experience. SJTs not only gather input from candidates but they also provide a vehicle to communicate messages about the role and the company. Images can demonstrate a diverse range of people in different roles.
2. Within interviews, SJTs can be incorporated as a vehicle to evaluate important capabilities where past experience may not be able to provide a true picture of longer term potential. This can play an important role in levelling out the playing field where some candidates are able to readily share examples having enjoyed the privilege of suitable stretch opportunities. For example where looking for examples of Strategic Thinking, it can be very difficult to share past examples unless you have already been operating in equivalent roles. Using SJT’s in an interview might challenge our old rules of avoiding the hypothetical “what would you do if” questions. However, where we are trying to evaluate the candidate’s thought process, supplementary questions can reveal useful evidence of thinking skills that goes beyond their ability to spot the text book answer.
3. Within an assessment centre context, there can be an important role for SJTs alongside the more typical interpersonal and task based exercises. Thought based competencies can be extremely challenging to observe and accurately evaluate through interactive role-plays. We inevitably jump to conclusions about the thoughts and motivations that lie behind observable behaviours and these assumptions will be impacted by stereotypes and bias. Meanwhile it can be uncomfortable to avoid reintroduce impersonal cognitive testing at this later stage that might measure reasoning ability and as we have already discussed reintroduce risks of adverse impact. Using SJTs or an SJT Interview provides a valuable objective measures of decision-making ability to complement the assessment of observable behaviour.
To talk more about how Pearn Kandola can help your organisation harness Situational Judgment Tests, please call or email James Meachin on 44 (0)1865 399060, jmeachin@pearnkandola.com
