Convergent and predictive validity of the big five factors assessed with singlestimulus and quasi-ipsative questionnaires

  1. Inmaculada Otero 1
  2. Dámaris Cuadrado 1
  3. Alexandra Martínez 1
  1. 1 University of Santiago de Compostela, Spain
Journal:
Revista de psicología del trabajo y de las organizaciones = Journal of work and organizational psychology

ISSN: 1576-5962

Year of publication: 2020

Volume: 36

Issue: 3

Pages: 215-222

Type: Article

DOI: 10.5093/JWOP2020A17 DIALNET GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Revista de psicología del trabajo y de las organizaciones = Journal of work and organizational psychology

Abstract

This research examines the convergent-discriminant and predictive validity of the Big Five personality dimensions assessed with two different formats of personality inventories: a single-stimulus (SS) and a quasi-ipsative forced-choice (FC). The relationship between both types of measures and intelligence (GMA) was also analyzed. The results showed that: (1) the SS and the quasi-ipsative FC measures present a high convergent-discriminant validity; (2) the Big Five personality dimensions, assessed with both questionnaires, and GMA are independent constructs; and (3) both types of personality measures have similar predictive validity for the three criteria examined (academic performance, training success, and interpersonal competence). As expected, conscientiousness was the best predictor of academic performance and training success. Extraversion was the best predictor of interpersonal competence. Finally, the theoretical and practical implications of these findings are discussed.

Bibliographic References

  • Adair, C. (2014). Interventions for addressing faking on personality assessments for employee selection: A meta-analysis (Unpublished doctoral dissertation). DePaul University, Chicago, IL.
  • Aguinis, H., Henle, C. A., & Ostroff, C. (2001). Measurement in work and organizational psychology. In N. Anderson, D. Z. Ones, H. K. Sinagil, & C. Viswesvaran (Eds.), Handbook of industrial, work and organizational psychology (pp. 27-50). Sage Publications.
  • Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44(1), 1-26. https://doi.org/10.1111/j.1744-6570.1991.tb00688.x
  • Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next? International Journal of Selection and Assessment, 9(1-2), 9-30. https://doi.org/10.1111/1468-2389.00160
  • Bartram, D. (2005). The Great Eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology, 90(6), 1185-1203. https://doi.org/10.1037/0021-9010.90.6.1185
  • Bartram, D. (2007). Increasing validity with forced-choice criterion measurement formats. International Journal of Selection and Assessment, 15(3), 263-272. https://doi.org/10.1111/j.1468-2389.2007.00386.x
  • Benet-Martínez, V., & John, O. P. (1998). Los “cinco grandes” across cultures and ethnic groups: Multitrait multimethod analyses of the Big Five in Spanish and English. Journal of Personality and Social Psychology, 75(3), 729-750. https://doi.org/10.1037/0022-3514.75.3.729
  • Berry, C. M., Ones, D. S., & Sackett, P. R. (2007). Interpersonal deviance, organizational deviance, and their common correlates: A review and meta-analysis. Journal of Applied Psychology, 92(2), 410-424. https://doi.org/10.1037/0021-9010.92.2.410
  • Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317-335. https://doi.org/10.1111/j.1468-2389.2006.00354.x
  • Borman, W. C., Penner, L. A., Allen, T. D., & Motowidlo, S. J. (2001). Personality predictors of citizenship performance. International Journal of Selection and Assessment, 9(1-2), 52-69. https://doi.org/10.1111/1468-2389.00163
  • Brown, A., & Maydeu-Olivares, A. (2013). How IRT can solve problems of ipsative data in forced-choice questionnaires. Psychological Methods, 18(1), 36-52. https://doi.org/10.1037/a0030641
  • Cao, M., & Drasgow, F. (2019). Does forcing reduce faking? A meta-analytic review of forced-choice personality measures in high-stakes situations. Journal of Applied Psychology, 104(11), 1347-1368. https://doi.org/10.1037/apl0000414
  • Caprara, G. V., & Perugini, M. (1994). Personality described by adjectives: The generalizability of the Big Five to the Italian lexical context. European Journal of Personality, 8(5), 357-369. https://doi.org/10.1002/per.2410080502
  • Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005). Reconsidering forced-choice item formats for applicant personality assessment. Human Performance, 18(3), 267-307. https://doi.org/10.1207/s15327043hup1803_4
  • Cohen, J. (1977). Statistical power analysis for the behavioral sciences. Academic Press.
  • Collins, J. M., Schmidt, F. L., Sanchez-Ku, M., Thomas, L., & McDaniel, M. (1999). Predicting assessment center ratings from cognitive ability and personality. Paper presented at the 14th Annual Conference of the SIOP, Atlanta, Georgia, April 29-May 2.
  • Collins, J. M., Schmidt, F. L., Sanchez–Ku, M., Thomas, L., McDaniel, M. A., & Le, H. (2003). Can basic individual differences shed light on the construct meaning of assessment center evaluations? International Journal of Selection and Assessment, 11(1), 17-29. https://doi.org/10.1111/1468-2389.00223
  • Connolly, J. J., Kavanagh, E. J., & Viswesvaran, C. (2007). The convergent validity between self and observer ratings of personality: A meta-analytic review. International Journal of Selection and Assessment, 15(1), 110-117. https://doi.org/10.1111/j.1468-2389.2007.00371.x
  • Converse, P. D., Oswald, F. L., Imus, A., Hedricks, C., Roy, R., & Butera, H. (2006). Forcing choices in personality measurement. In R. L. Griffith, & M. H. Peterson (Eds.), A closer examination of applicant faking behavior (pp. 263-282). Information Age.
  • Costa, P. T., & McCrae, R. R. (1992). Neo personality inventory-revised (NEO-PI-R) and neo five-factor inventory (NEO-FFI) professional manual. Psychological Assessment Resources.
  • Cuadrado, D., Salgado, J., & Moscoso, S. (2020). Counterproductive academic behaviors and academic performance: A meta-analysis and structural equation model. Manuscript submitted for publication.
  • DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology, 93(5), 880-896. https://doi.org/10.1037/0022-3514.93.5.880
  • Digman, J. M., & Takemoto-Chock, N. K. (1981). Factors in the natural language of personality: Re-analysis and comparison of six major studies. Multivariate Behavioral Research, 16(2), 149-170. https://doi.org/10.1207/s15327906mbr1602_2
  • Ferrando, P. J., & Lorenzo-Seva, U. (2014). Exploratory item factor analysis: Additional considerations. Anales de Psicología, 30(3), 1170-1175. https://doi.org/10.6018/analesps.30.3.199991
  • Goldberg, L. R. (1990). An alternative” description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, 59(6), 1216-1229. https://doi.org/10.1037/0022-3514.59.6.1216
  • Goldberg, L. R. (1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4(1), 26-42. https://doi.org/10.1037/1040-3590.4.1.26
  • Hicks, L. E. (1970). Some properties of ipsative, normative, and forced-choice normative measures. Psychological Bulletin, 74(3), 167-184. https://doi.org/10.1037/h0029780
  • Horn, J. L. (1971). Motivation and dynamic calculus concepts from multivariate experiment. In R. B. Cattell (Ed.), Handbook of multivariate experimental psychology (2nd printing, pp. 611-641). Tand McNally.
  • Hough, L. M. (1992). The Big Five personality variables-construct confusion: Description versus prediction. Human Performance, 5(1-2), 139-155. https://doi.org/10.1080/08959285.1992.9667929
  • Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96(1), 72-98. https://doi.org/10.1037/0033-2909.96.1.72
  • Hurtz, G. M., & Donovan, J. J. (2000). Personality and job performance: The Big Five revisited. Journal of Applied Psychology, 85(6), 869-879. https://doi.org/10.1037/0021-9010.85.6.869
  • Jackson, D. N., Wroblewski, V. R., & Ashton, M. C. (2000). The impact of faking on employment tests: Does forced choice offer a solution? Human Performance, 13(4), 371-388. https://doi.org/10.1207/S15327043HUP1304_3
  • Joubert, T., Inceoglu, I., Bartram, D., Dowdeswell, K., & Lin, Y. (2015). A comparison of the psychometric properties of the forced choice and Likert scale versions of a personality instrument. International Journal of Selection and Assessment, 23(1), 92-97. https://doi.org/10.1111/ijsa.12098
  • Judge, T. A., Jackson, C. L., Shaw, J. C., Scott, B. A., & Rich, B. L. (2007). Self-efficacy and work-related performance: the integral role of individual differences. Journal of Applied Psychology, 92(1), 107-127. https://doi.org/10.1037/0021-9010.92.1.107
  • Kuncel, N. R., Hezlett, S. H., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the graduate record examinations: Implications for graduate student selection and performance. Psychological Bulletin, 127(1), 162-181. https://doi.org/10.1037/0033-2909.127.1.162
  • Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2004). Academic performance, career potential, creativity, and job performance: Can one construct predict them all? Journal of Personality and Social Psychology, 86(1), 148-161. https://doi.org/10.1037/0022-3514.86.1.148
  • Lee, P., Lee, S., & Stark, S. (2018). Examining validity evidence for multidimensional forced choice measures with different scoring approaches. Personality and Individual Differences, 123, 229-235. https://doi.org/10.1016/j.paid.2017.11.031
  • Lim, B. C., & Ployhart, R. E. (2006). Assessing the convergent and discriminant validity of Goldberg’s International Personality Item Pool: A multitrait-multimethod examination. Organizational Research Methods, 9(1), 29-54. https://doi.org/10.1177/1094428105283193
  • Lloret-Segura, S., Ferreres-Traver, A., Hernández-Baeza, A., & Tomás-Marco, I. (2014). El análisis factorial exploratorio de los ítems: una guía práctica, revisada y actualizada. Anales de Psicología, 30(3), 1151-1169. https://doi.org/10.6018/analesps.30.3.199361
  • Martínez, A. (2019). Evaluación empírica de un modelo teórico de los efectos del “faking” sobre las medidas de personalidad ocupacional [Empirical assessment of a theoretical model of the effects of faking on the scores of occupational personality] (Unpublished doctoral dissertation). University of Santiago de Compostela, Santiago de Compostela, Spain.
  • Morillo, D., Abad, F. J., Kreitchmann, R. S., Leenen, I., Hontangas, P., & Ponsoda, V. (2019). The journey from Likert to forced-choice questionnaires: Evidence of the invariance of item parameters. Journal of Work and Organizational Psychology, 35(2), 75-83. https://doi.org/10.5093/jwop2019a11
  • Mount, M. K., & Barrick, M. R. (1995). The Big Five personality dimensions: Implications for research and practice in human resources management. In K. M. Rowland, & G. Ferris (Eds.), Research in personnel and human resources management (Vol. 13, pp.153-200). JAI Press.
  • Murphy, K. R. (2005). Why don’t measures of broad dimensions of personality perform better as predictors of job performance? Human Performance, 18(4), 343-357. https://doi.org/10.1207/s15327043hup1804_2
  • Nguyen, N. T., & McDaniel, M. A. (2000, December). Brain size and intelligence: A meta-analysis. Paper presented at the 1ª Annual Conference of the International Society of Intelligence Research, Cleveland, OH.
  • Norman, W. T. (1963). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Social Psychology, 66(6), 574-583.
  • Ones, D. S., Viswesvaran, C., & Dilchert, S. (2005). Cognitive ability in personnel selection decisions. In A. Evers, O. Voskuijl, & N. Anderson (Eds.), The Blackwell handbook of selection (pp. 143-173). Blackwell.
  • Organ, D. W., & Ryan, K. (1995). A meta-analytic review of attitudinal and dispositional predictors of organizational citizenship behavior. Personnel Psychology, 48(4), 775-802. https://doi.org/10.1111/j.1744-6570.1995.tb01781.x
  • Peabody, D., & Goldberg, L. R. (1989). Some determinants of factor structures from personality-trait descriptors. Journal of Personality and Social Psychology, 57(3), 552-567. https://doi.org/10.1037/0022-3514.57.3.552
  • Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322-338. https://doi.org/10.1037/a0014996
  • Poropat, A. E. (2014). A meta-analysis of adult-rated child personality and academic performance in primary education. British Journal of Educational Psychology, 84(2), 239-252. https://doi.org/10.1111/bjep.12019
  • Postlethwaite, B. E. (2011). Fluid ability, crystallized ability, and performance across multiple domains: A meta-analysis (Doctoral dissertation). The University of Iowa, Iowa. https://doi.org/10.17077/etd.zopi8wvs
  • Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353-387. https://doi.org/10.1037/a0026838
  • Rosse, J. G., Stecher, M. D., Miller, J. L, & Levin, R. A. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83(4), 634-644. https://doi.org/10.1037/0021-9010.83.4.634
  • Salgado, J. F. (1997). The Five Factor Model of personality and job performance in the European Community. Journal of Applied Psychology, 82(1), 30-43. https://doi.org/10.1037/0021-9010.82.1.30
  • Salgado, J. F. (1998a). Big Five personality dimensions and job performance in army and civil occupations: A European perspective. Human Performance, 11(2-3), 271-288. https://doi.org/10.1080/08959285.1998.9668034
  • Salgado, J. F. (1998b). Manual técnico del inventario de personalidad de cinco factores (IP/5F) [Manual of the Five Factors of Personality Inventory IP/5F]. Santiago de Compostela, Spain: Tórculo.
  • Salgado, J. F. (2002). The Big Five personality dimensions and counterproductive behaviors. International Journal of Selection and Assessment, 10(1-2), 117-125. https://doi.org/10.1111/1468-2389.00198
  • Salgado, J. F. (2003). Predicting job performance using FFM and non-FFM personality measures. Journal of Occupational and Organizational Psychology, 76(3), 323-346. https://doi.org/10.1348/096317903769647201
  • Salgado, J. F. (2004, April). Moderator effects of job complexity on the Big Five validity. Poster presented at the 19th Annual Conference of the Society for Industrial and Organizational Psychology, Chicago, IL.
  • Salgado, J. F. (2010). Escalas de desempeño académico: CDTE, CDCE y CDAN [Academic performance scales: CDTE, CDCE, and CDAN] (Unpublished manuscript). Department of Social and Basic Psychology and Methodology, University of Santiago de Compostela, Santiago de Compostela, Spain.
  • Salgado, J. F. (2014). Reliability, construct, and criterion validity of the Quasi-Ipsative Personality Inventory (QI5F/Tri). (Unpublish manuscript.) University of Santiago de Compostela, Santiago de Compostela, Spain.
  • Salgado, J. F. (2016). A theoretical model of psychometric effects of faking on assessment procedures: Empirical findings and implications for personality at work. International Journal of Selection and Assessment, 24(3), 209-228. https://doi.org/10.1111/ijsa.12142
  • Salgado, J. F. (2017). Moderator effects of job complexity on the validity of forced-choice personality inventories for predicting job performance. Journal of Work and Organizational Psychology, 33(3), 229-238. https://doi.org/10.1016/j.rpto.2017.07.001
  • Salgado, J. F., Anderson, N., & Táuriz, G. (2015). The validity of ipsative and quasi-ipsative forced-choice personality inventories for different occupational groups: A comprehensive meta-analysis. Journal of Occupational and Organizational Psychology, 88(4), 797-834. https://doi.org/10.1111/joop.12098
  • Salgado, J. F., & Lado, M. (2018). Faking resistance of a quasi-ipsative forced-choice personality inventory without algebraic dependence. Journal of Work and Organizational Psychology, 34(3), 213-216. https://doi.org/10.5093/jwop2018a23
  • Salgado, J. F., & Moscoso, S. (2019). The validity of general mental ability for five performance criteria: Hunter and Hunter (1984) revisited. Frontiers in Psychology, 10, 2227. https://doi.org/10.3389/fpsyg.2019.02227
  • Salgado, J. F., Moscoso, S., & Anderson, N. (2013). Personality and counterproductive work behavior. In N. D. Christiansen & R. P. Tett (Eds.), Handbook of personality at work (pp. 606-632). Routledge.
  • Salgado, J. F., Moscoso, S., & Lado, M. (2003). Evidence of cross-cultural invariance of the big five personality dimensions in work settings. European Journal of Personality, 17(51), 67-76. https://doi.org/10.1002/per.482
  • Salgado, J. F., & Táuriz, G. (2014). The Five-Factor Model, forced-choice personality inventories and performance: A comprehensive meta-analysis of academic and occupational validity studies. European Journal of Work and Organizational Psychology, 23(1), 3-30. https://doi.org/10.1080/1359432X.2012.716198
  • Salgado, J. F., Viswesvaran, C., & Ones, D. S. (2001). Predictors used for personnel selection: An overview of constructs, methods and techniques. In N. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), Handbook of industrial, work and organizational psychology. Vol. 1. Personnel psychology (p. 165-199). Sage Publications Ltd.
  • Saucier, G., & Goldberg, L. R. (1998). What is beyond the Big Five? Journal of Personality, 66(4), 495-524. https://doi.org/10.1111/1467-6494.00022
  • Saucier, G., & Goldberg, L. R. (2002). Assessing the Big Five: Applications of 10 psychometric criteria to the development of marker scales. In B. de Raad & M. Perugini (Eds.), Big Five assessment (pp. 30-54). Hogrefe & Huber Publishers.
  • Scarpello, V. G., Ledvinka, J., & Bergmann, T. J. (1995). Human Resource Management. South-Western College Publishing.
  • Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings. Sage.
  • Scholz, G., & Schuler, H. (1993). Das nomologische Netzwerk des Assessment Centers: Eine Metaanalyse, [The nomological network of the assessment center: A meta-analysis]. Zeitschrift für Arbeits-und Organisationspsychologie, 37(2), 73-85.
  • Trapmann, S., Hell, B., Hirn, J. O. W., & Schuler, H. (2007). Meta-analysis of the relationship between the Big Five and academic success at university. Zeitschrift für Psychologie / Journal of Psychology 215(2), 132-151. https://doi.org/10.1027/0044-3409.215.2.132
  • Vasilopoulos, N. L., Cucina, J. M., Dyomina, N. V., Morewitz, C. L., & Reilly, R. R. (2006). Forced-choice personality tests: A measure of personality and cognitive ability? Human Performance, 19(3), 175-199. https://doi.org/10.1207/s15327043hup1903_1
  • Viswesvaran, C., & Ones, D. S. (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59(2), 197-210. https://doi.org/10.1177/00131649921969802
  • Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81(5), 557-574. https://doi.org/10.1037/0021-9010.81.5.557
  • Wonderlic (1992). Wonderlic Personnel Test & Scholastic Level Exam. User’s manual. Wonderlic, Inc.