Corrections for criterion reliability in validity generalizationThe consistency of hermes, the utility of midas

  1. Jesús F. Salgado
  2. Silvia Moscoso
  3. Neil Anderson
Revista:
Revista de psicología del trabajo y de las organizaciones = Journal of work and organizational psychology

ISSN: 1576-5962

Ano de publicación: 2016

Volume: 32

Número: 1

Páxinas: 17-23

Tipo: Artigo

DOI: 10.1016/J.RPTO.2015.12.001 DIALNET GOOGLE SCHOLAR lock_openAcceso aberto editor

Outras publicacións en: Revista de psicología del trabajo y de las organizaciones = Journal of work and organizational psychology

Resumo

En la literature se critica el uso de los coeficientes interjueces para corregir por la fiabilidad del criterio en los estudios de generalización de la validez (GV) y cuestionan si .52 es un estimador preciso y no dudoso de la fiabilidad interjueces de las valoraciones del desempeno˜ global en el trabajo. En este articulo, presentamos un meta-análisis de segundo orden de tres estudios meta-analíticos independientes sobre la fiabilidad interjueces de las valoraciones del desempeno˜ en el trabajo y hacemos diversos comentarios y reflexiones sobre el artículo de LeBreton et al. Los resultados de nuestro meta-análisis indican que la fiabilidad interjueces es .52 (k = 66, N = 18.582, SD = .105) para un único supervisor. Nuestras principales conclusiones son: (a) el valor de .52 es un estimador preciso de la fiabilidad interjueces del desempeno˜ global en el trabajo para un único valorador, (b) no es razonable concluir que los estudios de GV que han usado .52 como valor de la fiabilidad del criterio tengan una fundamentación estadística poco segura, (c) sobre la base de la fiabilidad interjueces, la fiabilidad test-retest y el coeficiente alfa, los juicios del supervisor son una medida útil y adecuada del desempeno˜ en el trabajo y pueden ser usados con confianza como criterio, (d) la corrección de la validez por falta de fiabilidad del criterio ha sido unánimemente recomendada por los psicómetras y psicólogos industriales “clásicos” como el método correcto de estimar la validez del predictor y es todavía recomendada en la actualidad y (e) la contribución sustantiva de los procedimientos de GV para orientar las prácticas de recursos humanos en las organizaciones no debería perderse en estas cuestiones técnicas de debate.

€ Ver financiamento

Información de financiamento

The research reported was partially supported by Grant PSI2014-56615-P from the Ministry of Economy and Competitiveness to Jesús F. Salgado and Silvia Moscoso and by a Leverhulme Trust grant number IN-2012-095 to Neil Anderson. Appendix Formulas used 1) Formula to attenuate the reliability coefficient due to range restriction (Formula of Otis-Kelley): = = unrestricted reliability coefficient. r y y = 1 − U 2 1 − R y y ; where U SD / sd and R yy 2) Formula to correct validity for range restriction (Thorndike's Case I; Pearson, 1902; Thondike,1949): = = observed validity; R = corrected validity. R 12 = 1 − u 2 2 1 − r 12 2 ; where u sd / SD ; r 12 12 3) Formula to correct validity for double range restriction ( Pearson, 1908 ): = = sd/SD in variable 2; = observed validity, and = validity corrected for double range restriction. R 12 = r 12 u 1 u 2 1 − r 12 2 1 − u 1 2 1 − r 12 2 1 − u 2 2 ; where u 1 sd / SD in variable 1, and u 2 r 12 R 12

Financiadores

Referencias bibliográficas

  • N.R. Anderson Relationships between practice and research in personnel selection: Does the left hand know what the right is doing? A. Evers, N. Anderson, O. Smit-Voskuyl (Eds.), Handbook of personnel selection 2005 Blackwell Oxford, UK 1 24
  • L.J. Cronbach Essentials of psychological testing Fifth edition 1990 Harper & Row New York, NY
  • E. Ghiselli, and C. Brown Personnel and industrial psychology 1948 McGraw-Hill New York, NY
  • E. Ghiselli, J.P. Campbell, and S. Zedeck Measurement theory for the behavioral sciences 1981 Freeman San Francisco, CA
  • J.P. Guilford Psychometric methods 1954 McGraw-Hill New York, NY
  • R.M. Guion Personnel testing 1965 McGraw-Hill New York, NY
  • R.M. Guion Assessment, measurement, and prediction for personnel decisions 1998 Lawrence Erlbaum Associates Mahwah, NJ
  • R.M. Guion, and S. Highouse Essentials of personnel assessment and selection 2006 Erlbaum Mahwah, NJ
  • H. Gulliksen Theory of mental tests 1950 Wiley New York, NY
  • J.E. Hunter, and R.F. Hunter Validity and utility of alternative predictors of job performance Psychological Bulletin 96 1984 72 98
  • T.L. Kelley The reliability of test scores Journal of Educational Research 3 1921 370 379
  • J.M. LeBreton, K.T. Scherer, and L.R. James Correction for criterion reliability in validity generalization: A false prophet in a land of suspended judgment Industrial and Organizational Psychology 7 2014 478 500
  • Q. McNemar Psychological statistics 3rd edition 1962 Wiley New York, NY
  • K.R. Murphy, and R. De Shon Interrater correlations do not estimate the reliability of job performance ratings Personnel Psychology 53 2000 873 900
  • J. Nunnally Psychometric methods 1978 McGraw-Hill New York, NY
  • A.S. Otis A method for inferring the change in a coefficient of correlation resulting from a change in the heterogeneity of the group Journal of Educational Psychology 13 1922 293 294
  • K. Pearson On the influence of double selection on the variation and correlation of two characters Biometrika 6 1908 111 112
  • Ronan, W.W., & Prien, E. (Eds.) (1971). Perspectives of the measurement of human performance. New York, NY: Appleton-Century Crofts.
  • H.R. Rothstein Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe Journal of Applied Psychology 75 1990 322 327
  • P.R. Sackett The status of validity generalization research: Key issues in drawing inferences from cumulative research studies K.R. Murphy (Ed.), Validity generalization: A critical review 2003 Lawrence Erlbaum Associates, Inc Mahwah, NJ 91 114
  • P.R. Sackett, D.J. Putka, and R.A. McCloy The concept of validity and the process of validation Neal Schmitt (Ed.) The Oxford handbook of personnel assessment and selection. 2012 Oxford University Press Oxford, UK
  • J.F. Salgado Estimating coefficients of equivalence and stability for job performance ratings: the importance of controlling for transient error on criterion measurement International Journal of Selection and Assessment 23 2015 37 44
  • J.F. Salgado, N. Anderson, S. Moscoso, C. Bertua, F. De Fruyt, and J.P. Rolland A meta-analytic study of general mental ability validity for different occupations in the European Community Journal of Applied Psychology 88 2003 1068 1081
  • J.F. Salgado, N. Anderson, and G. Tauriz The validity of ipsative and quasi-ipsative forced-choice personality inventories for different occupational groups: A comprehensive meta-analysis Journal of Occupational and Organizational Psychology 88 2015 797 834
  • Salgado, J. F., Bastida, M., Vázquez, S., & Moscoso, S. (manuscript under review). Happiness, positive emotions and job performance: a four-year longitudinal study
  • J.F. Salgado, and S. Moscoso Meta-analysis of interrater reliability of job performance ratings in validity studies of personnel selection Perceptual and Motor Skills 83 1996 1195 1201
  • J.F. Salgado, and G. Tauriz The five-factor model, forced-choice personality inventories and performance: A comprehensive meta-analysis of academic and occupational validity studies European Journal of Work and Organizational Psychology 23 2014 3 30
  • F.L. Schmidt, and J.E. Hunter Measurement error in psychological research: Lessons from 26 research scenarios Psychological Methods 1 1996 199 223
  • F.L. Schmidt, and J.E. Hunter The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings Psychological Bulletin 124 1998 262 274
  • F.L. Schmidt, H. Le, and I. Remus Beyond Alpha: an empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs Psychological Methods 8 2003 206 224
  • N. Schmitt, and R. Klimoski Research methods in human resources management 1991 South-Western Publishing Co Cincinnati, OH
  • B. Schneider, and N. Schmitt Staffing organizations 1986 Scott, Foresman Glenview, IL
  • R.L. Thorndike Personnel selection. Test and measurement techniques 1949 Wiley New York, NY
  • C. Viswesvaran, D. Ones, and F.L. Schmidt Comparative analysis of the reliability of job performance ratings Journal of Applied Psychology 81 1996 557 574
  • C. Viswesvaran, F.L. Schmidt, and D.S. Ones The moderating influence of job performance dimensions on convergence of supervisor and peer ratings of job performance: Unconfounding construct-level convergence and rating difficulty Journal of Applied Psychology 87 2002 345 354
Fonte de datos: Dialnet