Estimación de habilidad y precisión en tests adaptativos informatizados y tests óptimosUn caso práctico

  1. Real Deus, José Eulogio
  2. Ponsoda Gil, Vicente
  3. Olea Díaz, Julio
  4. Abad García, Francisco José
Journal:
REMA

ISSN: 1135-6855

Year of publication: 2002

Volume: 7

Issue: 1

Type: Article

DIALNET GOOGLE SCHOLAR lock_openDialnet editor

More publications in: REMA

Abstract

Sometimes, in applied contexts, there is needed to choose among Computerized Adaptive Test (CAT) or adapted to the subject and OptimalTests (OT) or adapted to the group. This problemwas analyzed for an English Vocabulary Test. Some unexplained group ability differences were found in a previous study (Olea, Revuelta, Ximénez and Abad, 2000); here we try to explain such differences as "error" (v.g.: bias) from the estimationprocedure. A program was developed to calculate the theoretical bias as functionof the abilitylevel(Lord, 1983; Samejima, 1993a). In the first place, the theoretical biaswas analyzed for the OT. Insecond place, the convergence betweenthe theoretical, simulated and empirical results was studied considering the CAT estimates as the less unbiased ones. The results showthat the differences between the CAT and the OT cannot be explained by the estimation procedure and they should be attributed to other causes.

Bibliographic References

  • 1. Baker, F.B. (1992): Item Response Theory. Parameter estimation techniques. New York:Marcel Dekker.
  • 2. Bock, R.D. & Aitkin, M. (1981): Marginal maximum likelihood estimation of item parameters: an application of an EM algorithm. Psykometrika, 46, 443-459.
  • 3. Bock, R.D. & Mislevy, R.J. (1982): Adaptative EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
  • 4. De Ayala, R.J., Schafer, W.D. & Sava-Bolesta, M. (1995):An investigationofthe standard errors of expected a posteriori ability estimates. British Journal of Mathematical and Statistical Psychology, 47, 385-405.
  • 5. Hambleton, R.K., Slater, S.C., Narayanan, P. ySetiadi, H. (1996):Construcciónautomatizada de los tests:conceptos básicos, avances técnicos y aplicaciones. En J. Muñiz (Coor.). Psicometría. Madrid: Universitas.
  • 6. Kim, J.K. & Nicewander, A.(1993).Abilityestimationfor conventionaltests. Psychometrika, 58 (4), 587-599.
  • 7. Lord, F. M. (1983). Unbiased estimators of abilityparameters, oftheir variance, and of their parallelforms reliability. Psychometrika, 48, 2, 233-245.
  • 8. Lord, F. M. (1984). Maximum likelihood and bayesian parameter estimation in item response theory. Educational Testing Service, Princeton, N.J.
  • 9. Lord, F.M. (1986). Maximum likelihood and Bayesian parameter estimation in itemresponse theory. Journal of Educational Measurement, 23, 157-162.
  • 10. Mislevy, R.J. & Bock, R. D. (1990). BILOG 3: item analysis and test scoring with binary logistic models. Mooresville, IN: Scientific-Software International.
  • 11. Muraki, E. & Bock, R.D.(1997). PARSCALE:IRTitem analysis and test scoring forrating-scale data. Chicago, IL: Scientific-Software International.
  • 12. Olea,J. & Ponsoda, V. (1996). Tests AdaptativosInformatizados. En J. Muñiz (Coor.).Psicometría. Madrid: Universitas
  • 13. Olea,J.;Ponsoda, V.;Revuelta,J.; Belchí, J. (1996). Propiedades psicométricas de untest adaptativo de vocabulario inglés. Estudios de Psicología, 55, 61-73.
  • 14. Olea, J., Revuelta, J., Ximénez, C. y Abad, F.J. (2000). Psychometric and psychological effects of review on computerized fixed and adaptive tests. Psicológica, 21(1-2), 157-173.
  • 15. Ponsoda, V.; Olea, J.; Revuelta, J. (1994). ADTEST: A computer-adaptive test based on the maximum information principle. Educational and Psychological Measurement, 54 (3), 680-686.
  • 16. Ponsoda, V.; Wise, S.L.; Olea, J.; Revuelta, J. (1997).An investigation of self-adapted testing in a Spanish high school population. Educational and Psychological Measurement, 57 (2), 210-221.
  • 17. Revuelta, J. y Ponsoda,V. (1997). Una solución a la estimación inicial en los tests adaptativos informatizados, R.E.M.A., 2(2), 1-6.
  • 18. Revuelta, J.; Ponsoda, V. (1998). A comparison of item exposure control methods in computerized adaptive testing. Journal of Educational Measurement, 35 (4), 311-327.
  • 19. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, 17.
  • 20. Samejima, F. (1993a). An approximation for the bias function of the maximum likelihood estimate of a latent variable for the general case where the item responses are discrete. Psychometrika, 58, 119-138.
  • 21. Samejima, F. (1993b). The bias function of the maximum likelihood estimate of ability for the dichotomous response level. Psychometrika, 58, 195-209.
  • 22. Segall,D.O.; Moreno, K.E.(1999). Development ofthe Computerized Adaptive Testing versionofthe Armed Services Vocational Aptitude Battery. En Drasgow, F.; Olson-Buchanan, J. (Eds.) Innovations in computerized assessment. Mahwah, NJ, USA: Lawrence Erlbaum Associates.
  • 23. Stroud, A.H. & Secherst, D. (1966). Gaussian quadrature formules. Englewood Cliffs, NJ: Prentice-Hall.
  • 24. Thissen, D. (1991).MULTILOG user’sguide: multiple, categorical item analysis and test scoring using item response theory. Chicago, IL: Scientific-Software International.
  • 25. Vispoel, W.P., Wang, T. & Bleiler, T. (1997): The efficiency, reliability, and concurrent validity of adaptative and fixed-itemmusic listening tests. Journal of Educational Measurement,34,43-63.
  • 26. Wang, T. (1997). Essentially unbiased estimates in computerized adaptive testing. Paper presented at the annual meeting of the AERA, Chicago.
  • 27. Wang, T.; Vispoel, W.P. (1998). Properties of ability estimation methods in computerized adaptive testing. Journal of Educational Measurement, 35(2), 109-135.
  • 28. Warm, A. W. (1989). Weigthed likelihood estimation of ability in item response theory with tests of finite length. Psychometrika, 54, 427-450.
  • 29. Wise, S.L. (1999). Tests autoadaptados informatizados: fundamentos, resultados de investigación e implicaciones para la aplicación práctica. EnOlea, J., Ponsoda, V. & Prieto, G. (Eds): Tests Informatizados. Fundamentos y aplicaciones. Madrid: Pirámide.
Data source: Dialnet