In the introduction to his new and eloquent publication examining the current state of psychometrics, Professor Peter Saville of Saville Consultants, one of the leading exponents of the science, declares: "While this book provides a broadly positive account of psychometric assessments, an appropriate amount of scepticism can be healthy and, we would argue, is important for educating people to recognise the good, the bad and the ugly within the world of psychometrics. It is a simple fact that not all psychometrics are born equal and it is regrettable that there are a number of charlatans peddling low-quality (and potentially unethical) products to credulous users."
Saville's word of warning is, perhaps, symptomatic of the fact that in recent years psychometrics has undergone something of a sea change, as Delia Pegg, a chartered psychologist and head of assessment at Odgers Berndtson's London office, explains: "I think compared with 10 years ago there are just so many instruments available and there is a real variety in terms of the level of quality; some of them claim to do absolutely everything. Some of the psychometric profiles will be tailored to specific roles or industries. What you get with some suppliers is that they will have one questionnaire or one main profile and from that you can generate lots of different narrower elements that are suited to, say, sales roles or to the entrepreneurial role or to leadership skills."
Pegg agrees that this expansion of the industry does have its potential downside: "If you lobbed everything now available at a candidate because you thought it might be interesting, they'd be sitting there doing psychometric questionnaires and tests all day." The art, says Pegg, lies in carefully selecting and adapting from the plethora of instruments out there. "We try to be more consultative by selecting the things that we think are best for our clients."
For Peter Saville the situation is equally clear: "Well-constructed psychometric assessments are shown to be among the most accurate and fair ways of assessing people in the workplace. My job is to reduce risk. I cannot give you 100 per cent certainty, but with relevant measures you can probably improve your selection by about 30 per cent - assuming you have good applicants to work from."
That is the nub of the matter. Good tests, carried out by good practitioners, will always play an important role in the selection process. Poor or ill-conceived tests (created by "charlatans") will only serve to confuse matters, tarnishing the reputation of a long-established, scientifically based industry and potentially leading to the wrong hire.
So what are these more established, robust tests? The origin (and basis) of almost all personality profiles can be traced back to what is known as the 'big five' theory of personality attributes: Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism.
Although psychometric testing has been in use for more than a century, in the past 20 or 30 years the term 'psychometrics' has broadened to include an array of aptitude tests, personality profiles and motivation and interest inventories, often through the deployment of instruments by such groundbreakers as Isabel Myers and Katharine Briggs who launched MBTI in 1962, Dr Robert Hogan who introduced the Hogan Personality Inventory in the 1970s, the aforementioned Peter Saville and Roger Holdsworth who launched OPQ in 1984, and the rise of the emotional intelligence questionnaires first introduced in the 1990s.
In addition, the next phase of psychometric testing may well revolve around the digital world we all now inhabit. Consider this extract from an article recently carried in Psyche, the newsletter of The Psychometrics Forum, written by Gemma Smith also of Saville Consulting: "Algorithms are already being developed that link 'big five' factors with social media: for example, TweetPsych provides a report of your personality based on qualitative text mining of your Twitter feeds."
Smith goes on to state that "in 10 years' time, 46 per cent of workers in the US will be Millennials, defined in part through an obsession with both themselves and technology. The psychometrics industry will clearly need to consider researching and developing more innovative ways of assessment and recruiting."
Andrew Munro from AM Azure, a business psychology consultancy, who is less than convinced of the efficacy of many popular non-cognitive-based psychometrics (see below), says of the potential alliance of testing with the digital space: "I talked to a colleague about this the other day and he believes there will be innovation - but it will not come from the current big players in the test publishing market but from software houses who are keen on big data and begin to do clever things in terms of how they analyse patterns as part of assessment processes. They know how the maths works and they can put in place very complex algorithms."
While the correlation between the digital space and testing evolves, practitioners continue to ensure that they use - or adapt - the best of the tests available. One take on the traditional Hogan approach has been adapted for use within a number of Odgers Berndtson's offices worldwide and forms the core of Leaderfit™ - a proprietary product developed by Eric Beaudan, Director of the firm's Global Leadership Practice, who is based in Toronto. Says Beaudan: "Leaderfit™ is a senior executive assessment method that provides a comparative view of leadership potential mapped to five dimensions of executive high performance (Strategic Clarity, Execution Savvy, Resilience & Adaptability, Stakeholder Management and People & Team Development). Adds Beaudan: "We have learned over time that leadership success is based on how effectively executives develop strategy and link it to execution, how they build teams and manage relationships, and how they manage themselves. We are now able to measure where executives fall on these key dimensions, which is ideal for individual selection or assessment, executive succession or similar team development."
Beaudan, like Delia Pegg and Peter Saville, is also of the opinion that testing has evolved. "In the past 10-15 years," he says, "it has become a little more commonplace to use testing all the way up to C-Suite-level selection, succession planning and development, and I think one of the reasons it's becoming more commonplace is that the tools have evolved significantly. Hogan now has many dimensions and is very effective and predictive of executive success. The field has evolved towards being globally accepted in terms of selecting and developing leaders."
Beaudan adds that the Leaderfit™ suite, which derives in great part from Hogan-based tools such as its grouping of 11 possible 'derailers' to executive success, has been key in the decision-making process for CEOs and senior-level roles. Derailers usually stem from a personality or leadership strength that on first glance is very attractive in a leader, but when amplified the same trait can actually lead an executive to fail. An example might include being too confident or arrogant, or too cautious or reserved.
Beaudan has been doing some fascinating work on, for example, measuring integrity: "There are particular scales in our Hogan-based instruments that will measure an individual's likelihood to commit fraud or to engage in behaviour that is, to use Hogan terminology, mischievous or questionable. So we can see character combinations that when put together act as a bit of a tripwire or warning that alerts us that there are likely to be behavioural issues that you might not see on the surface." Beaudan says that clients' needs have been evolving too. "Ten years ago about 10 per cent of the companies I worked with were using psychometric testing," he says, "whereas today that number is closer to 40 per cent." According to Beaudan, the significant number of leadership failures within large organisations during that time has taught boards to do their due diligence when hiring at leadership level. "They don't just want to hire the person who can do the job," he says "they want to hire someone who is going to be a good fit with the organisation - and fit is very hard to determine from an interview."
But Beaudan and his colleagues do have their detractors. Andrew Munro is one of them: "We know from research that 30 per cent of any population will fake a personality test. The chances, therefore, of picking up a successful psychopathic faker are pretty remote. I'm just not convinced that these tests pick up the 'dark side', the toxic type of leader. Last year I wrote a paper called 'Predictive Stall' in which I said that while psychometric testing historically has delivered, unfortunately these potential gains might be washed out through falling standards and poor practitioner practice such as failing to optimise test data in selection decision-making.
"The stall is also explained," continues Munro, "because applicants have shifted from compliant candidates to shrewd game players. This argument suggests that test publishers are being outmanoeuvred by a generation of applicants adept enough in social media to exchange experiences about interview methodology, assessment centre protocols or the 'Goldilocks' tactics of personality test completion. It also points to an escalating 'arms race' in which any innovation by the test publishers is countered quickly by applicants who find stratagems to mitigate their effect."
Munro sums up his analysis thus: "It seems that psychometric testing is stuck and needs to revisit either its predictive claims or the research programmes it is using to build evidence of its predictive power in response to the selection challenges of the 21st century."
Though Saville disagrees, he says that: "In practice there are modern methods for detecting cheating, and people usually feel that what they endorse in a questionnaire is what they should be. A lot can be achieved with the use of carefully selected questions."
And Eric Beaudan is even more direct. "It's very hard at the end of the day when you've got psychometric data, 360 data, structured interview data, for somebody to fool us into believing that they're a good fit for a particular executive role or organisation when they're not."
The psychometric testing industry is, then, going through a period of change and realignment. Ultimately, though, it is the quality of the candidate that matters; psychometric testing has never been anything other than one valuable part of the process that ensures the leaders of the next generation are the right ones.
In the second part of our series, Keiron Pim explores how to look after yourself when the pressur...
Chapter one of our ‘Leadership, Disrupted’ report revealed how technologically-fuelled disruption...