The Effectiveness of Computerized Adaptive Testing in Estimating Mental Ability Using Raven's Matrices

Ahmad S. Odeh, Omar S. Obaidat

Abstract


The purpose of this study was to investigate the effectiveness of computerized adaptive testing on the accuracy of estimation of mental ability using Raven matrices based on two estimation methods (MLE versus MAP) and based on the test completion rule (limited number of items versus least standard error) . To achieve this purpose, item bank was built consisting of 105 items drawn from these matrices. Five computerized tests were used in the study, applied on a sample consisting of 638 students. Results of the study indicated that the rule of ending test with a limited number of items provide more accurate mental ability estimations, and more accurate information function than least standard error rule. Least standard error rule is 50% more accurate than ending tests with a limited number of items. Computerized adaptive testing provides more accurate mental ability estimations, and reduces the number of administered items by 70%. Furthermore, it has higher information function than linear testing depending on the used one of the two methods of estimation. Both MLE and MAP give equal ability estimations and accuracy indices, but MLE has a higher information function than MAP. Based on the results of the study it was recommended to: use adaptive testing for different practical purposes and to use maximum likelihood method in estimating ability based on adaptive testing, more studies were recommended to compare the two estimation methods with different numbers of items in the bank and with different lowest level standard error.

Keywords


Computerized Adaptive Testing, Raven Matrices, Item Response Theory, Item Bank, Adaptive Testing Completion Rule, Ability Estimation Accuracy.

Full Text:

PDF

Refbacks

  • There are currently no refbacks.