Evaluation of validity and validation by means of the argument-based approach
Extended abstract – Saskia Wools
Validity is the most important quality aspect of tests and assessments, but it is not clear how validity can be evaluated. This article presents a procedure for the evaluation of validity and validation which is an extension of the argument-based approach to validation. Within the argument-based approach two phases are distinguished: the development stage in which an interpretive argument is built and an appraisal stage for the construction of a validity argument. For the evaluation of this validation process a third stage is added: an evaluation stage. The evaluation stage consists of the application of criteria to evaluate the interpretive argument, the validity evidence provided, and the validity argument. This procedure is illustrated with an existing assessment: the driver performance assessment. As part of this illustration the argument-based approach is applied to validate the driver performance assessment. Therefore, both an interpretive argument and a validity argument are constructed. Subsequently, the criteria are applied to evaluate these arguments.
The article concludes with recommendations for the application of the procedure. The first recommendations relate to the application of the argument-based approach. It seemed that construction an interpretive argument is very complicated and therefore it is recommended that it should be developed by a team that consists of content experts as well as measurement experts. This team should also be aware of the strength of validity evidence that is already produced during the development of an assessment. Furthermore, it appeared that the argument-based approach can play a guiding role in validation research. For improving the evaluation stage it is recommended that software is developed to guide the evaluation process. Furthermore, it is suggested that future research should aim for defining what evidence is sufficient for establishing validity. Finally, it might be interesting to investigate the possibilities of using the argument-based approach to validation as a framework for the evaluation of tests and assessments in general.