Quasi-ipsative Forced-Choice Personality Inventories and the Control of FakingThe Biasing Effects of Transient Error

  1. Alexandra Martínez 1
  2. Jesús F. Salgado 1
  3. Mario Lado 1
  1. 1 University of Santiago de Compostela, Spain
Revista:
Revista de psicología del trabajo y de las organizaciones = Journal of work and organizational psychology

ISSN: 1576-5962

Ano de publicación: 2022

Volume: 38

Número: 3

Páxinas: 241-248

Tipo: Artigo

DOI: 10.5093/JWOP2022A16 DIALNET GOOGLE SCHOLAR lock_openAcceso aberto editor

Outras publicacións en: Revista de psicología del trabajo y de las organizaciones = Journal of work and organizational psychology

Resumo

La investigación experimental sobre los efectos del faking o falseamiento en las medidas de personalidad ha utilizado dos tipos de diseños: diseños intrasujeto y diseños entre sujetos. Pero ninguno de ellos nos permite controlar los efectos del error temporal en el faking. Usando un diseño de cuadrado latino (DCL), este estudio examina los efectos del faking en los Cinco Grandes evaluados con un inventario de personalidad de elección forzosa (EF) cuasi-ipsativo. El DCL es un diseño experimental que simultáneamente nos permite controlar las diferencias entre sujetos, la variabilidad intrasujeto y el error temporal. La muestra estuvo compuesta por 246 participantes (cuatro grupos experimentales, evaluados dos veces en un intervalo de 2-3 semanas). Los resultados mostraron que (1) el tamaño del efecto del faking se puede atribuir en gran medida a un error temporal y (2) el formato de EF causi-ipsativo muestra una gran resistencia al faking. El tamaño del efecto promedio (d de Cohen) para los Cinco Grandes fue 0.21, 0.12 y 0.09 para el faking observado, el error temporal y el faking verdadero, respectivamente. En promedio, el 62 % del tamaño del efecto del faking observado se puede atribuir a un error temporal. Para concluir, se discuten las implicaciones de estos resultados.

Referencias bibliográficas

  • Abad, F. J., Kreitchmann, R. S., Sorrel, M. A., Nájera, P., García-Garzón, E., Garrido, L. E., & Jiménez, M. (2022). Building adaptive forced choice tests “On the Fly” for personality measurement. Psychologist Papers, 43(1), 29-35. https://doi.org/10.23923/pap.psicol.2982
  • Adair, C. (2014). Interventions for addressing faking on personality assessments for employee selection: A meta-analysis (Unpublished doctoral dissertation). DePaul University.
  • Baron, H. (1996). Strengths and limitation of ipsative measurement. Journal of Occupational and Organizational Psychology, 69(1), 49-56. https://doi.org/10.1111/j.2044-8325.1996.tb00599.x
  • Bartram, D. (1996). The relationship between ipsatized and normative measures of personality. Journal of Occupational and Organizational Psychology, 69(1), 25-39. https://doi.org/10.1111/j.2044-8325.1996.tb00597.x
  • Becker, G. (2000). How important is transient error in estimating reliability? Going beyond simulation studies. Psychological Methods, 5(3), 370-379. https://doi.org/10.1037/1082-989X.5.3.370
  • Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317-335. https://doi.org/10.1111/j.1468-2389.2006.00354.x
  • Brown, A., & Maydeu-Olivares, A. (2013). How IRT can solve problems of ipsative data in forced-choice questionnaires. Psychological Methods, 18(1), 36-52. https://doi.org/10.1037/a0030641
  • Cao, M., & Drasgow, F. (2019). Does forcing reduce faking? A meta-analytic review of forced-choice personality measures in high-stakes situations. Journal of Applied Psychology, 104(11), 1347-1368. https://doi.org/10.1037/apl0000414
  • Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005). Reconsidering forced-choice item formats for applicant personality assessment. Human Performance, 18(3), 267-307. https://doi.org/10.1207/s15327043hup1803_4
  • Clemans, W. V. (1966). An analytical and empirical examination of some properties of ipsative measures. Psychometric Monographs, 14, 1-56.
  • Cochran, W. G., & Cox, G. M. (1978). Experimental designs. John Wiley & Sons, Inc.
  • Converse, P. D., Oswald, F. L., Imus, A., Hedricks, C., Roy, R., & Butera, H. (2006). Forcing choices in personality measurement. In R. L. Griffith & M. H. Peterson (Eds.), A closer examination of applicant faking behavior (pp. 263-282). Information Age.
  • Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Houghton Mifflin.
  • Dilchert, S., & Ones, D. S. (2012). Measuring and improving environmental sustainability. In S. E. Jackson, D. S. Ones, & S. Dilchert (Eds.), Managing HR for environmental sustainability (pp. 187-221). Jossey-Bass/Wiley.
  • Donovan, J. J., Dwight, S. A., & Schneider, D. (2014). The impact of applicant faking on selection measures, hiring decisions, and employee performance. Journal of Business and Psychology, 29, 479-493. https://doi.org/10.1007/s10869-013-9318-5
  • Douglas, E. F., MacDaniel, M. A., & Snell, A. F. (1996). The validity of non-cognitive measures decays when applicants fake. Academy of Management Proceedings, 1996(1), 127-131. https://doi.org/10.5465/ambpp.1996.4979062
  • Dunlap, W, P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1(2), 170-177. https://doi.org/10.1037/1082-989X.1.2.170
  • Ellingson, J. E., Sackett, P. R., & Connelly, B. S. (2007). Personality assessment across selection and development contexts: Insights into response distortion. Journal of Applied Psychology, 92(2), 386-395. https://doi.org/10.1037/0021-9010.92.2.386
  • Fisher, P. A., Robie, C., Christiansen, N. D., Speer, A. B., & Schneider, L. (2019). Criterion-related validity of forced-choice personality measures: A cautionary note regarding Thurstonian IRT versus classical test theory scoring. Personnel Assessment and Decisions, 5(1), 49-61. https://doi.org/10.25035/pad.2019.01.003
  • Gao, L. (2005). Latin squares in experimental design. Unpublished manuscript. Michigan State University.
  • García-Izquierdo, A. L., Ramos-Villagrasa, P. J., & Lubiana, M. A. (2020). Developing biadata for public selection purposes: A comparison between fuzzy logic and traditional methods. Journal of Work and Organizational Psychology, 36(3) 231-242. https://doi.org/10.5093/jwop2020a22
  • Grant, D. A. (1948). The latin square principle in the design and analysis of psychological experiments. Psychological Bulletin, 45(5), 427-442. https://doi.org/10.1037/h0053912
  • Griffith, R. L., Chmielowski, T., & Yoshita, Y. (2007). Do applicants fake? An examination of the frequency of applicant faking behavior. Personnel Review, 36(3), 341-355. https://doi.org/10.1108/00483480710731310
  • Griffith, R. L., & Converse, P. D. (2012). The rules of evidence and the prevalence of applicant faking. In M. Ziegler, C. MacCann, & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 34-52). Oxford University Press.
  • Heggestad, E. D., George, E., & Reeve, C. L. (2006). Transient error in personality scores: Considering honest and faked responses. Personality and Individual Differences, 40(6), 1201-1211. https://doi.org/10.1016/j.paid.2005.10.014
  • Hicks, L. E. (1970). Some properties of ipsative, normative, and forced-choice normative measures. Psychological Bulletin, 74(3), 167-184. https://doi.org/10.1037/h0029780
  • Hooper, A. C. (2007). Self-presentation on personality measures in lab and field settings: A meta-analysis (Unpublished doctoral dissertation). University of Minnesota.
  • Horn, J. L. (1971). Motivation and dynamic calculus concepts from multivariate experiment. In R. B. Cattell (Ed.), Handbook of multivariate experimental psychology (2nd printing, pp. 611-641). Tand McNally.
  • Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75(5), 581-595. https://doi.org/10.1037/0021-9010.75.5.581
  • Jackson, D. N., Wroblewski, V. R., & Ashton, M. C. (2000). The impact of faking on employment tests: Does forced choice offer a solution? Human Performance, 13(4), 371-388. https://doi.org/10.1207/S15327043HUP1304_3
  • Kirk, R. E. (2013). Experimental design: Procedures for the behavioral sciences (4th Edition). Sage.
  • Komar, S., Brown, D. J., Komar, J. A., & Robie, C. (2008). Faking and the validity of conscientiousness: A Monte Carlo investigation. Journal of Applied Psychology, 93(1), 140-154. https://doi.org/10.1037/0021-9010.93.1.140
  • MacCann, C., Ziegler, M., & Roberts, R. D. (2012). Faking in personality assessment: Reflections and recommendations. In M. Ziegler, C. MacCann, & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 309-329). Oxford University Press.
  • Martínez, A. (2019). Evaluación empírica de un modelo teórico de los efectos del faking sobre las medidas de personalidad ocupacional [Empirical assessment of a theoretical model of the effects of faking on the scores of occupational personality] (Unpublished doctoral dissertation). University of Santiago de Compostela.
  • Martínez, A., Moscoso, S., & Lado, M. (2021). Effects of faking on the predictive validity of a quasi-ipsative forced-choice personality inventory: Implications for sustainable personnel selection. Sustainability, 13(8), 4398. https://doi.org/10.3390/su13084398
  • Martínez, A., Moscoso, S., & Lado, M. (2021). Faking effects on the factor structure of a quasi-ipsative forced-choice personality inventory. Journal of Work and Organizational Psychology, 37(1), 1-10. https://doi.org/jwop2021a7
  • Martínez, A., & Salgado, J. F. (2021). A meta-analysis of the faking resistance of forced-choice personality inventories. Frontiers in Psychology, 12, Article 732241. https://doi.org/10.3389/fpsyg.2021.732241
  • Meade, A. W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77(4), 531-552. https://doi.org/10.1348/0963179042596504
  • Mesmer-Magnus, J., & Viswesvaran, C. (2006). Assessing response distortion in personality tests: A review pf research designs and analytic strategies. In R. L. Griffith & M. H. Peterson (Eds.), A closer examination of applicant faking behavior (pp. 85-114). Information Age.
  • Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K., & Schmitt, N. (2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60(3), 683-729. https://doi.org/10.1111/j.1744-6570.2007.00089.x
  • Morillo, D., Abad, F. J., Kreitchmann, R. S., Leenen, I., Hontangas, P., & Ponsoda, V. (2019). The journey from Likert to forced-choice questionnaires: Evidence of the invariance of item parameters. Journal of Work and Organizational Psychology, 35(2), 75-83. https://doi.org/10.5093/jwop2019a11
  • Murphy, K. R. (2005). Why don’t measures of broad dimensions of personality perform better as predictors of job performance? Human Performance, 18(4), 343-357. https://doi.org/10.1207/s15327043hup1804_2
  • Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81(6), 660-679. https://doi.org/10.1037/0021-9010.81.6.660
  • Otero, I., Cuadrado, D., & Martínez, A. (2020). Convergent and predictive validity of the Big Five Factors assessed with single stimulus and quasi-ipsative questionnaires. Journal of Work and Organizational Psychology, 36(3), 215-222. https://doi.org/10.5093/jwop2020a17
  • Rothstein, M. G., & Goffin, R. D. (2006). The use of personality measures in personnel selection: What does current research support? Human Resource Management Review, 16(2), 155-180. https://doi.org/10.1016/j.hrmr.2006.03.004
  • Salgado, J. F. (2014). Reliability, construct, and criterion validity of the Quasi-Ipsative Personality Inventory (QI5F/Tri). Unpublished manuscript. University of Santiago de Compostela.
  • Salgado, J. F. (2016). A theoretical model of psychometric effects of faking on assessment procedures: Empirical findings and implications for personality at work. International Journal of Selection and Assessment, 24(3), 209-228. https://doi.org/10.1111/ijsa.12142
  • Salgado, J. F. (2017). Moderator effects of job complexity on the validity of forced-choice personality inventories for predicting job performance. Journal of Work and Organizational psychology, 33(3), 229-238. https://doi.org/10.1016/j.rpto.2017.07.001
  • Salgado, J. F., Anderson, N., & Táuriz, G. (2015). The validity of ipsative and quasi-ipsative forced-choice personality inventories for different occupational groups: A comprehensive meta-analysis. Journal of Occupational and Organizational Psychology, 88(4), 797-834. https://doi.org/10.1111/joop.12098
  • Salgado, J. F., & Lado, M. (2018). Faking resistance of a quasi-ipsative forced-choice personality inventory without algebraic dependence. Journal of Work and Organizational Psychology, 34(3), 213-216. https://doi.org/10.5093/jwop2018a23
  • Salgado, J. F., & Táuriz, G. (2014). The Five-Factor Model, forced-choice personality inventories and performance: A comprehensive meta-analysis of academic and occupational validity studies. European Journal of Work and Organizational Psychology, 23(1), 3-30. https://doi.org/10.1080/1359432X.2012.716198
  • Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. Psychological Methods, 8(2), 206-224. https://doi.org/10.1037/1082-989X.8.2.206
  • Smith, D. B., & McDaniel, M. (2012). Questioning old assumptions: Faking and the personality-performance relationship. In M. Ziegler, C. MacCann, & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 53-70). Oxford University Press.
  • Vautier, S., & Jmel, S. (2003), Transient error or specificity? An alternative to the staggered equivalent split-half procedure. Psychological Methods, 8(2), 225-238. https://doi.org/10.1037/1082-989X.8.2.225
  • Viswesvaran, C., & Ones, D. S. (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59(2), 197-210. https://doi.org/10.1177/00131649921969802
  • Zickar, M. J., & Gibby, R. E. (2006). A history of faking and socially desirable responding on personality tests. In R. L. Griffith & M. H. Peterson (Eds.), A closer examination of applicant faking behavior (pp. 21-42). Information Age.
  • Ziegler, M., MacCann, C., & Roberts, R. D. (2012). Faking: knowns, unknowns, and points of contention. In M. Ziegler, C. MacCann, & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 3-16). Oxford University Press.
Fonte de datos: Dialnet