Dealing with multiple objectives in test assembly (B. Veldkamp en M. Matteucci)

Bent ual geabonneerd op de nieuwsbrief?







Dealing with multiple objectives in test assembly



Bernard P Veldkamp

Research Center for Examination and Certification

University of Twente - The Netherlands


Mariagiulia Matteucci

Statistics Department

University of Bologna - Italy





Requests for information can be send to: Bernard Veldkamp, University of Twente, PO box 217, 7500 AE Enschede, The Netherlands. E-mail: b.p.veldkamp@utwente.nl

Abstract

It is quite common that tests or exams are being used for more then one purpose. For example, they are used to measure the ability of the students in a reliable manner. At the same time, they are used for pass/fail decisions or to predict future behavior of the candidate. The question remains how to assemble a test that can be used for all these different purposes, that is, how to assemble a multi-objective test. Besides, multiple objectives can result from different purposes, but also from the way test specifications have been implemented. For the WDM-model, for multidimensional IRT, for Cognitive Diagnostic CAT, but also for infeasibility analysis, multiple objective test assembly problems have to be solved.

In this paper, a 2-stage method is presented for dealing with multiple objectives in test assembly. In the normalization stage, all objectives are brought on a common scale. In the valorization stage, the different objectives are being compared and related to each other. The method is applied to a Guidance Test developed at the University of Bologna, and a comparison is made with more traditional single objective test assembly methods. The results clearly demonstrate the importance and relevance of multi-objective test assembly.


Keywords: automated test assembly, multiple criteria, multiple objectives, simulated annealing, guidance test.