After last week’s blog there was a lot of comments offline to the effect:
‘Great blog. What are ipsative tests?’
Ipsative testing, or rather the critique of, has been something that has in many ways been done to death in I/O psychology. However, as is the case with most things, history has a tendency to repeat as a new generation, unaware of the past are doomed to make the same mistakes.
The arguments for and against ipsative testing are both simple and complex. At a simple level, it is theoretically unsound as a means of comparing people as it is a within-person measurement. At a complex level, the arguments are around factor structure, reliability and in-built bias. This has resulted in the definitive paper using OPQ32 data (Meade, 2004) concluding:
In sum, the standards required of tests used for employee selection are quite strict with regards to validity and reliability of the selection instruments. As such, the limitations inherent with ipsative measures pose too great a threat to the validity of the selection tools to make it a useful instrument for selection on a trait-by-trait basis.
But, I’m getting ahead of myself! In this three-part blog, I will briefly discuss the basis for ipsative measurement, the issues with ipsative testing and the arguments brought forth in support. In keeping with my commitment to context and history I will do this through distilling previous discussions I have had in the past in public forums on the topic:
What is ipsative testing?
Ipsative testing is a technique derived in the 1940s to look at within-person measurement. Cattell (1944, as cited from Hammond and Barrett, 1995) introduced the term ‘ipsative’, and defined it as measurement relative to other measures within the individual. Ipsative scores reflect only relative strengths of traits within the individual. An ipsative scale uses the behaviour of the individual to create its own standard. For example, a patient’s condition may be viewed as having either improved or declined relative to the patient’s own average or relative condition.
In contrast, normative scales measure absolute differences, and reflect an underlying continuum common across all people, as in measures of IQ. Normative tests measure quantifiable characteristics on individual scales. These scales can vary independently. Also, the scores can measure such characteristics of an individual against confirmed patterns of statistical normality (e.g. bell curve). Such testing allows people to be compared to particular groups, populations, or jobs. These tests can also be used as developmental and training tools.
How does ipsative testing work and what does it look like?
Ipsative tests are often known as forced choice measures. People are asked to state what is ‘most like them’ and what is ‘least like them’ from a range of options. This creates a hierarchy of choice inside an individual showing the relative strengths and weaknesses of trait scores within an individual.
What can ipsative testing be used for; what do the best practice guidelines state?
By its very design, ipsative tools are designed for counselling or individual coaching where the focus is the individual, not comparisons. Because the scores derived from the measures are relative to the individual and not independent of each other, scale scores cannot be compared across people. The best practice guidelines are derived from the broad range of references cited dating from the 1950’s to 2001. The reason the guidelines don’t change is that they reflect how ipsative tools work (within rather than between measurements) and what they should and should not be used for.
BPS guidelines for Level B Psychometric Training
6.11: Explain in non-technical terms the main issues involved in the ‘ipsative-normative’ debate concerning when ipsative measures should and should not be used.
In summary, ipsative testing is very applicable when working with an individual. Ipsative testing is inappropriate when used for selection.
Cattell, R.B. (1944) Psychological Measurement: ipsative, normative, and interactive. Psychological Review, 51, 292-303.
Meade, A.W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77, 531–552
Hammond, S., and Barrett, P., (1995). The Psychometric and Practical Implications of the use of Ipsative, forced-choice format, Questionnaires.
Hugh, L.M., and Ones. D. S., (2001). The structure, measurement, validity and use of Personality variables in Industrial, Work and Organisational Psychology, 233.