Evaluation of knowledge with multiple-choice tests: three of four options? Experience with admission examinations to medical and surgical postgraduate studies at University of Antioquia (Medellín, Colombia)
DOI:
https://doi.org/10.17533/udea.iatreia.v28n3a08Keywords:
educational measurement, item analysis, multiple choice questionsAbstract
Introduction: The aim of this study was to evaluate the effect of reducing the number of response options per question on the psychometric indicators of an exam for admission to postgraduate medical studies, at University of Antioquia, in Medellín, Colombia.
Methodology: Application of psychometric assessment indexes from the perspective or two theories: the classical of measurement and the item response, to a test of 70 questions, applied in 2014 to 2.539 candidates. The least frequently chosen distractor was eliminated and randomly replaced by one of the three remaining ones.
Results: Only 52.9% of the questions had three functional distractors. No difference was found in the difficulty, discrimination, standard error of measurement, Cronbach's alpha and the coefficient of biserial correlation (classical measurement theory). Also, there was no difference in the extent of item difficulty or ability of people (item response theory). The test with three options retained a good fit.
Conclusion: Multiple choice tests with three response options performed as well as their four options counterparts.
Downloads
References
(1.) Gimeno Sacristán J. La evaluación en la enseñanza. En: Sacristán Gimeno J, Gómez Pérez AI, editores. Comprender y transformar la enseñanza. 5ª ed. Madrid: Morata; 1996. p. 334-97.
(2.) Haladyna TM, Downing SM, Rodriguez MC. A Review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ. 2002 Jul;15(3):309-33.
(3.) García-Garro AJ, Ramos-Ortega G, Díaz de León-Ponce MA, Olvera-Chávez A. Instrumentos de evaluación. Rev Mex Anestesiol. 2007;30(3):158-64.
(4.) Moreno R, Martínez RJ, Muñiz J. Directrices para la construcción de ítems de elección múltiple. Psicothema. 2004;16(3):490-7.
(5.) Rodriguez MC. Three options are optimal for multiple-choice items: a meta-analysis of 80 years of research. Educ Meas Issues Pract. 2005 Jun;24(2):3-13.
(6.) Tarrant M, Ware J, Mohammed AM. An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis. BMC Med Educ. 2009 Jul;9:40.
(7.) Vyas R, Supe A. Multiple choice questions: a literature review on the optimal number of options. Natl Med J India. 2008;21(3):130-3.
(8.) Rogausch A, Hofer R, Krebs R. Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: a simulation and survey. BMC Med Educ. 2010 Jan;10(1):85.
(9.) Kasule OH. Overview of medical student assessment: Why, what, who, and how. J Taibah Univ Med Sci. 2013 Aug;8(2):72-9.
(10.) Morrison S, Free KW. Writing multiple-choice test items that promote and measure critical thinking. J Nurs Educ. 2001 Jan;40(1):17-24.
(11.) Brady AM. Assessment of learning with multiple-choice questions. Nurse Educ Pract. 2005 Jul;5(4):238-42.
(12.) McMahan CA, Pinckard RN, Prihoda TJ, Hendricson WD, Jones AC. Improving multiple-choice questions to better assess dental student knowledge: distractor utilization in oral and maxillofacial pathology course examinations. J Dent Educ. 2013 Dec;77(12):1593-609.
(13.) Matlock-Hetzel S. Basic Concepts in Item and Test Analysis [Internet]. En: Annual meeting of the Southwest Educational Research Association; Austin, January, 1997. Texas: Texas A&M University; 1997 [consultado 2014 Abr 15]. Disponible en: http://ericae.net/ft/tamu/Espy.htm
(14.) Backhoff Escudero E, Larrazolo Reyna N, Rosas Morales M. Nivel de dificultad y poder de discriminación del Examen de Habilidades y Conocimientos Básicos (EXHCOBA). REDIE. 2000;2(1):12-29.
(15.) Mitra NK, Nagaraja HS, Ponnudurai G, Judson JP. The levels of difficulty and discrimination indices in type a multiple choice questions of pre-clinical semester 1, multidisciplinary summative tests. IeJSME. 2009;3(1):2-7.
(16.) Downing SM. Reliability: on the reproducibility of assessment data. Med Educ. 2004 Sep;38(9):1006-12.
(17.) Subkoviak MJ. A Practitioner’s Guide to computation and interpretation of reliability indices for mastery tests. J Educ Meas. 1988 Mar;25(1):47-55.
(18.) Altman DG, Bland JM. Measurement in Medicine : the analysis of method comparison studies. Statistician. 1983 Sep;32(3):307-17.
(19.) Shizuka T, Takeuchi O, Yashima T, Yoshizawa K. A comparison of three-and four-option English tests for university entrance selection purposes in Japan. Lang Test. 2006 Jan;23(1):35-57.
(20.) Baghaei P, Amrahi N. The effects of the number of options on the psychometric characteristics of multiple choice items. Psychol Test Assess Model. 2011;53(2):192-211.
(21.) Frary RB. More multiple-choice item writing do ́s and dont ́s. Pract Assess Res Eval. 1995;4(11):1-6.
(22.) Swanson DB, Holtzman KZ, Allbee K. Measurement characteristics of content-parallel single-best-answer and extended-matching questions in relation to num-ber and source of options. Acad Med. 2008 Oct;83(10 Suppl):S21-4.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2015 Iatreia
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Papers published in the journal are available for use under the Creative Commons license, specifically Attribution-NonCommercial-ShareAlike 4.0 International.
The papers must be unpublished and sent exclusively to the Journal Iatreia; the author uploading the contribution is required to submit two fully completed formats: article submission and authorship responsibility.
